2024-03-08T08:15:39.012 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-08T08:15:39.012 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin} --archive /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506 --verbose --owner scheduled_jcollin@teuthology --name jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi -- /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/orig.config.yaml 2024-03-08T08:15:39.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T08:15:39.139 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506 branch: main description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin} email: jcollin@redhat.com first_in_suite: false job_id: '7585506' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: lockdep: true ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - missing required features sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-jcollin-test_per-client-perf-counters sha1: f6f80007dcf9e039bb253dd50a92ba3c71c6943a owner: scheduled_jcollin@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7204 sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-jcollin-test_per-client-perf-counters suite_path: /home/teuthworker/src/github.com_joscollin_ceph_f6f80007dcf9e039bb253dd50a92ba3c71c6943a/qa suite_relpath: qa suite_repo: https://github.com/joscollin/ceph.git suite_sha1: f6f80007dcf9e039bb253dd50a92ba3c71c6943a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKTYHwn2ct8CPgIFzcYR/P//Mbb19XWmlaFVPA0yaGWofg+DIDzwNQl46giMKuzPAIQPFxspymm1LHkkDBYnbCw= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqi72noN/IPdTZxLSwikM+xHAuFgsGYAZR4z2w1+1k2ffghjOr7RGkIowApJ83KB6CWEXWcIZfnN+TuORTPdEY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_admin.TestLabeledPerfCounters teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-08_08:09:40 tube: smithi user: jcollin verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-08T08:15:39.139 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_joscollin_ceph_f6f80007dcf9e039bb253dd50a92ba3c71c6943a/qa; will attempt to use it 2024-03-08T08:15:39.140 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_joscollin_ceph_f6f80007dcf9e039bb253dd50a92ba3c71c6943a/qa/tasks 2024-03-08T08:15:39.140 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-08T08:15:39.142 INFO:teuthology.task.internal:Checking packages... 2024-03-08T08:15:39.167 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1fb5d10608dccee73edd6ef8a2c7b804a097c324' 2024-03-08T08:15:39.168 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-08T08:15:39.168 INFO:teuthology.packaging:ref: None 2024-03-08T08:15:39.168 INFO:teuthology.packaging:tag: None 2024-03-08T08:15:39.168 INFO:teuthology.packaging:branch: main 2024-03-08T08:15:39.168 INFO:teuthology.packaging:sha1: 1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T08:15:39.168 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-03-08T08:15:39.516 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1940.g639d1827 2024-03-08T08:15:39.517 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-08T08:15:39.527 INFO:teuthology.task.internal:no buildpackages task found 2024-03-08T08:15:39.527 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-08T08:15:39.550 INFO:teuthology.task.internal:Saving configuration 2024-03-08T08:15:39.567 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-08T08:15:39.577 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T08:15:39.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 08:10:45.152895', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKTYHwn2ct8CPgIFzcYR/P//Mbb19XWmlaFVPA0yaGWofg+DIDzwNQl46giMKuzPAIQPFxspymm1LHkkDBYnbCw='} 2024-03-08T08:15:39.627 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 08:10:45.154648', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqi72noN/IPdTZxLSwikM+xHAuFgsGYAZR4z2w1+1k2ffghjOr7RGkIowApJ83KB6CWEXWcIZfnN+TuORTPdEY='} 2024-03-08T08:15:39.627 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-08T08:15:39.636 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-08T08:15:39.636 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-08T08:15:39.636 INFO:teuthology.run_tasks:Running task console_log... 2024-03-08T08:15:39.733 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f384ae1b0d0>, signals=[15]) 2024-03-08T08:15:39.733 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-08T08:15:39.744 INFO:teuthology.task.internal:Opening connections... 2024-03-08T08:15:39.744 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:15:39.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:15:39.860 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:15:39.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:15:39.954 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-08T08:15:39.964 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-03-08T08:15:39.981 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-03-08T08:15:39.981 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-03-08T08:15:40.040 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-03-08T08:15:40.040 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-03-08T08:15:40.040 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-03-08T08:15:40.040 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T08:15:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T08:15:40.042 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-03-08T08:15:40.061 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-03-08T08:15:40.078 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-03-08T08:15:40.079 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-03-08T08:15:40.137 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-03-08T08:15:40.137 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T08:15:40.138 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T08:15:40.139 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-03-08T08:15:40.161 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-08T08:15:40.173 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-08T08:15:40.185 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-08T08:15:40.185 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-03-08T08:15:40.189 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-03-08T08:15:40.204 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-08T08:15:40.216 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-08T08:15:40.216 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-03-08T08:15:40.245 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-03-08T08:15:40.337 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-08T08:15:40.390 INFO:teuthology.run_tasks:Running task kernel... 2024-03-08T08:15:40.409 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-08T08:15:40.409 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-03-08T08:15:40.410 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-08T08:15:40.410 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T08:15:40.410 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T08:15:40.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:15:40.427 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-03-08T08:15:40.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:15:40.428 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-03-08T08:15:40.483 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-361.el9.x86_64 2024-03-08T08:15:40.483 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.14.0-361.el9.x86_64 2024-03-08T08:15:40.484 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2024-03-08T08:15:40.485 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-361.el9.x86_64 2024-03-08T08:15:40.485 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.14.0-361.el9.x86_64 2024-03-08T08:15:40.485 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2024-03-08T08:15:42.742 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-08T08:15:43.168 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 6.4 MB/s | 6.7 MB 00:01 2024-03-08T08:15:44.158 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-08T08:15:44.593 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2024-03-08T08:15:45.399 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-03-08T08:15:45.742 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.0 MB 00:00 2024-03-08T08:15:47.617 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-08T08:15:47.710 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-08T08:15:51.935 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.1 MB 00:00 2024-03-08T08:15:53.311 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 15 kB 00:00 2024-03-08T08:15:54.404 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-08T08:15:54.886 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.1 MB 00:03 2024-03-08T08:15:56.218 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 15 kB 00:00 2024-03-08T08:15:57.170 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-03-08T08:15:59.379 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-08T08:16:01.594 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-08T08:16:01.594 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-08T08:16:01.684 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-08T08:16:01.692 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-08T08:16:01.692 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-03-08T08:16:01.692 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 100 M 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 123 M 2024-03-08T08:16:01.693 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-08T08:16:02.145 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-03-08T08:16:03.997 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): kernel-core-5.14.0-427.el9.x86_64.rpm 11 MB/s | 21 MB 00:01 2024-03-08T08:16:04.360 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-08T08:16:04.360 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-08T08:16:04.458 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-08T08:16:04.470 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-08T08:16:04.470 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2024-03-08T08:16:04.470 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout:Install 4 Packages 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:16:04.471 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 100 M 2024-03-08T08:16:04.472 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 123 M 2024-03-08T08:16:04.472 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-08T08:16:05.480 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 11 MB/s | 39 MB 00:03 2024-03-08T08:16:06.005 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): kernel-5.14.0-427.el9.x86_64.rpm 1.5 MB/s | 6.0 MB 00:03 2024-03-08T08:16:06.314 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 14 MB/s | 33 MB 00:02 2024-03-08T08:16:06.314 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-08T08:16:06.314 INFO:teuthology.orchestra.run.smithi144.stdout:Total 22 MB/s | 100 MB 00:04 2024-03-08T08:16:07.194 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-08T08:16:07.582 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-08T08:16:07.582 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-08T08:16:09.230 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-08T08:16:09.230 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-08T08:16:12.149 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-08T08:16:12.835 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-08T08:16:12.964 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T08:16:13.729 INFO:teuthology.orchestra.run.smithi105.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 667 kB/s | 6.0 MB 00:09 2024-03-08T08:16:15.793 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T08:16:15.977 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T08:16:20.715 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T08:16:20.895 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:16:25.664 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:16:35.902 INFO:teuthology.orchestra.run.smithi105.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 680 kB/s | 21 MB 00:31 2024-03-08T08:16:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:16:54.711 INFO:teuthology.orchestra.run.smithi144.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T08:16:54.711 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:16:54.794 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:16:55.248 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:16:55.248 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-08T08:16:55.248 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T08:16:55.248 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T08:16:55.711 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:16:55.712 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:16:55.712 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-08T08:16:55.712 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-08T08:16:55.712 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-08T08:16:55.712 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-08T08:16:55.712 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-08T08:16:55.712 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:16:55.712 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-08T08:16:56.193 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T08:16:56.351 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T08:16:56.353 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T08:16:56.353 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-08T08:16:56.353 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-08T08:16:56.353 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-08T08:16:56.353 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-08T08:16:56.353 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-03-08T08:16:56.353 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T08:16:56.474 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T08:16:56.474 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T08:16:56.475 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 68.el9 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8264254 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T08:16:56.512 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:16:56.513 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T08:16:56.514 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T08:16:56.514 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 5.14.0-427.el9.x86_64 2024-03-08T08:16:56.515 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T08:16:57.134 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-03-08T08:16:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T08:16:58.604 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-03-08T08:16:58.607 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-03-08T08:16:58.622 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.ZZ1D6Mv7Bk 2024-03-08T08:16:58.622 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZZ1D6Mv7Bk 2024-03-08T08:16:58.692 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.ZZ1D6Mv7Bk 2024-03-08T08:16:58.833 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.ZZ1D6Mv7Bk is 6KB 2024-03-08T08:16:58.882 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.ZZ1D6Mv7Bk 2024-03-08T08:16:58.911 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T08:16:58.960 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T08:16:58.961 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T08:16:58.961 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T08:16:58.961 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-08T08:16:58.962 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-08T08:16:59.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-03-08T08:17:04.916 INFO:teuthology.orchestra.run.smithi105.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 668 kB/s | 33 MB 00:51 2024-03-08T08:17:08.708 INFO:teuthology.orchestra.run.smithi105.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 627 kB/s | 39 MB 01:04 2024-03-08T08:17:08.708 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-08T08:17:08.708 INFO:teuthology.orchestra.run.smithi105.stdout:Total 1.5 MB/s | 100 MB 01:04 2024-03-08T08:17:09.596 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-08T08:17:09.976 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-08T08:17:09.976 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-08T08:17:11.324 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-08T08:17:11.325 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-08T08:17:14.249 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-08T08:17:14.894 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-08T08:17:15.025 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T08:17:17.831 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T08:17:18.003 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T08:17:22.794 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T08:17:22.989 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:17:27.749 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:17:29.111 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-08T08:17:29.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-08T08:17:29.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:17:47.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-03-08T08:17:56.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-08T08:17:56.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:17:57.007 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:17:57.008 INFO:teuthology.orchestra.run.smithi105.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T08:17:57.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:17:57.091 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:17:57.531 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:17:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-08T08:17:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-08T08:17:57.532 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-08T08:17:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-08T08:17:57.969 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:17:57.970 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-03-08T08:17:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-08T08:17:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-08T08:17:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-08T08:17:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-08T08:17:57.970 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:17:57.970 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-08T08:17:58.511 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T08:17:58.667 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T08:17:58.668 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T08:17:58.668 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-08T08:17:58.668 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-08T08:17:58.668 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-08T08:17:58.668 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-08T08:17:58.668 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-03-08T08:17:58.668 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-03-08T08:17:58.788 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-08T08:17:58.789 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-08T08:17:58.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2024-03-08T08:17:58.823 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2024-03-08T08:17:58.823 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2024-03-08T08:17:58.823 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.06 2024-03-08T08:17:58.823 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 68.el9 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Group : Unspecified 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 8264254 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : builder@centos.org 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:17:58.824 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T08:17:58.825 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T08:17:58.825 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T08:17:58.825 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2024-03-08T08:17:58.825 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:17:58.825 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T08:17:58.826 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T08:17:58.826 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 5.14.0-427.el9.x86_64 2024-03-08T08:17:58.826 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T08:17:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-03-08T08:17:59.618 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-03-08T08:18:00.897 INFO:teuthology.orchestra.run.smithi105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T08:18:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-03-08T08:18:00.927 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-03-08T08:18:00.942 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.ZREYH4BZE7 2024-03-08T08:18:00.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZREYH4BZE7 2024-03-08T08:18:01.012 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.ZREYH4BZE7 2024-03-08T08:18:01.154 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.ZREYH4BZE7 is 6KB 2024-03-08T08:18:01.202 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.ZREYH4BZE7 2024-03-08T08:18:01.216 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T08:18:01.289 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T08:18:01.290 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T08:18:01.290 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T08:18:01.290 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-08T08:18:01.291 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-08T08:18:01.452 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-03-08T08:18:11.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-08T08:18:11.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:18:31.474 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-08T08:18:31.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-08T08:18:31.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:18:43.515 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-08T08:18:43.939 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-08T08:18:43.939 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-427.el9.x86_64"... 2024-03-08T08:18:43.940 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-03-08T08:18:43.953 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-427.el9.x86_64 2024-03-08T08:18:43.954 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-08T08:18:43.954 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T08:18:43.954 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T08:18:44.955 DEBUG:teuthology.parallel:result is None 2024-03-08T08:18:50.017 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-03-08T08:18:59.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-08T08:18:59.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:19:02.078 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-03-08T08:19:14.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-08T08:19:14.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:19:46.012 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-08T08:19:46.428 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-08T08:19:46.429 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-08T08:19:46.429 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-03-08T08:19:46.444 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-427.el9.x86_64 2024-03-08T08:19:46.444 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-08T08:19:46.445 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T08:19:46.445 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T08:19:47.445 DEBUG:teuthology.parallel:result is None 2024-03-08T08:19:47.446 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-08T08:19:47.453 INFO:teuthology.task.internal:Creating test directory... 2024-03-08T08:19:47.454 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T08:19:47.456 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T08:19:47.475 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-08T08:19:47.483 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-08T08:19:47.491 INFO:teuthology.task.internal:Creating archive directory... 2024-03-08T08:19:47.492 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T08:19:47.509 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T08:19:47.589 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-08T08:19:47.596 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-08T08:19:47.597 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T08:19:47.599 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T08:19:47.680 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T08:19:47.716 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T08:19:47.748 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T08:19:47.856 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T08:19:47.857 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-08T08:19:47.865 INFO:teuthology.task.internal:Configuring sudo... 2024-03-08T08:19:47.865 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T08:19:47.867 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T08:19:47.899 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-08T08:19:47.909 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-08T08:19:47.909 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T08:19:47.933 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T08:19:47.956 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T08:19:48.024 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T08:19:48.071 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T08:19:48.153 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T08:19:48.191 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:19:48.191 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T08:19:48.259 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T08:19:48.308 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T08:19:48.357 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T08:19:48.442 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T08:19:48.487 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:19:48.487 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T08:19:48.557 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-03-08T08:19:48.559 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-03-08T08:19:48.604 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T08:19:48.654 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T08:19:49.075 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-08T08:19:49.087 INFO:teuthology.task.internal:Starting timer... 2024-03-08T08:19:49.088 INFO:teuthology.run_tasks:Running task pcp... 2024-03-08T08:19:49.101 INFO:teuthology.run_tasks:Running task selinux... 2024-03-08T08:19:49.111 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-03-08T08:19:49.208 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-03-08T08:19:49.210 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-03-08T08:19:49.264 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-03-08T08:19:49.266 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-08T08:19:49.266 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-03-08T08:19:49.291 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-03-08T08:19:49.292 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-03-08T08:19:49.316 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-03-08T08:19:49.316 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2024-03-08T08:19:49.317 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T08:19:49.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:19:49.357 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T08:19:49.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:19:49.387 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-08T08:19:49.388 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-03-08T08:19:49.452 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-03-08T08:19:49.501 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-08T08:19:49.517 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-08T08:19:49.540 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-08T08:19:49.541 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-08T08:24:03.848 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-03-08T08:24:03.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-08T08:24:03.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:24:03.941 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-08T08:24:04.017 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-08T08:24:04.017 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-08T08:24:04.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T08:24:04.110 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-08T08:24:04.189 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-08T08:24:04.189 INFO:teuthology.run_tasks:Running task clock... 2024-03-08T08:24:04.201 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-08T08:24:04.201 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T08:24:04.202 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T08:24:04.204 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T08:24:04.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T08:24:04.233 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T08:24:04.249 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T08:24:04.260 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T08:24:04.277 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T08:24:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-03-08T08:24:04.292 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-03-08T08:24:04.306 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-03-08T08:24:04.308 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T08:24:04.320 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-03-08T08:24:04.325 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T08:24:04.336 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T08:24:04.352 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T08:24:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-08T08:24:04.376 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T08:24:04.376 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-08T08:24:04.377 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:24:04.377 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:24:04.377 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:24:04.377 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:24:04.405 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-08T08:24:04.408 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T08:24:04.409 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-08T08:24:04.409 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:24:04.409 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:24:04.409 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:24:04.409 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:24:04.410 INFO:teuthology.run_tasks:Running task install... 2024-03-08T08:24:04.418 DEBUG:teuthology.task.install:project ceph 2024-03-08T08:24:04.419 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}} 2024-03-08T08:24:04.419 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'} 2024-03-08T08:24:04.419 INFO:teuthology.task.install:Using flavor: default 2024-03-08T08:24:04.424 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-08T08:24:04.425 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-08T08:24:04.425 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'tag': None, 'wait_for_package': False} 2024-03-08T08:24:04.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T08:24:04.427 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'tag': None, 'wait_for_package': False} 2024-03-08T08:24:04.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T08:24:04.571 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/ 2024-03-08T08:24:04.571 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1670.g1fb5d106 2024-03-08T08:24:04.697 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T08:24:04.697 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:24:04.697 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T08:24:04.725 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T08:24:04.725 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1fb5d10608dccee73edd6ef8a2c7b804a097c324/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T08:24:04.754 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/ 2024-03-08T08:24:04.754 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1670.g1fb5d106 2024-03-08T08:24:04.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T08:24:04.865 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/1fb5d10608dccee73edd6ef8a2c7b804a097c324/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T08:24:04.866 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:24:04.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T08:24:04.873 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T08:24:04.894 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T08:24:04.895 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1fb5d10608dccee73edd6ef8a2c7b804a097c324/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T08:24:04.938 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-03-08T08:24:04.940 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-03-08T08:24:04.963 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T08:24:05.042 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T08:24:05.109 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-03-08T08:24:05.111 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-03-08T08:24:05.175 INFO:teuthology.orchestra.run.smithi105.stdout:74 files removed 2024-03-08T08:24:05.202 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T08:24:05.353 INFO:teuthology.orchestra.run.smithi144.stdout:74 files removed 2024-03-08T08:24:05.380 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T08:24:06.326 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 101 kB/s | 82 kB 00:00 2024-03-08T08:24:06.551 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 96 kB/s | 82 kB 00:00 2024-03-08T08:24:07.102 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 14 kB/s | 10 kB 00:00 2024-03-08T08:24:07.276 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-03-08T08:24:07.661 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-03-08T08:24:07.893 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-03-08T08:24:08.680 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 6.8 MB/s | 6.7 MB 00:00 2024-03-08T08:24:08.829 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 7.5 MB/s | 6.7 MB 00:00 2024-03-08T08:24:10.414 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 3.2 MB/s | 2.0 MB 00:00 2024-03-08T08:24:10.487 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-08T08:24:11.662 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-03-08T08:24:12.804 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-03-08T08:24:21.745 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:08 2024-03-08T08:24:27.754 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 19 MB 00:13 2024-03-08T08:24:28.065 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.1 MB 00:02 2024-03-08T08:24:29.396 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 15 kB 00:00 2024-03-08T08:24:31.871 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-08T08:24:32.364 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 21 MB 00:02 2024-03-08T08:24:33.339 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 15 kB 00:00 2024-03-08T08:24:37.389 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 7.6 kB/s | 1.7 kB 00:00 2024-03-08T08:24:39.927 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T08:24:39.927 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T08:24:39.941 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T08:24:39.942 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T08:24:39.943 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T08:24:39.945 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T08:24:39.947 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T08:24:39.949 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T08:24:39.951 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T08:24:40.030 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-08T08:24:40.041 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 6.6 k 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.3 M 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 887 k 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 145 k 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 1.5 M 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 138 k 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 1.8 M 2024-03-08T08:24:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 7.4 M 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 49 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 9.8 M 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 47 M 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 264 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 748 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 220 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 27 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 18 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 31 k 2024-03-08T08:24:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 745 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 127 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 162 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 320 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 298 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 99 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 85 k 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T08:24:40.044 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 170 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T08:24:40.045 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.4 M 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T08:24:40.046 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 20 M 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 31 k 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 2.1 M 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 248 k 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 4.7 M 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 17 M 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 15 k 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 25 k 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T08:24:40.047 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 163 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T08:24:40.048 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 501 k 2024-03-08T08:24:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T08:24:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.1 M 2024-03-08T08:24:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T08:24:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T08:24:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T08:24:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T08:24:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T08:24:40.049 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T08:24:40.050 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 45 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 131 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T08:24:40.051 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T08:24:40.052 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T08:24:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T08:24:40.054 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:24:40.055 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-08T08:24:40.056 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-08T08:24:40.056 INFO:teuthology.orchestra.run.smithi144.stdout:Install 150 Packages 2024-03-08T08:24:40.056 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 24 Packages 2024-03-08T08:24:40.056 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:24:40.056 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 223 M 2024-03-08T08:24:40.056 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-08T08:24:40.890 INFO:teuthology.orchestra.run.smithi144.stdout:(1/174): ceph-19.0.0-1670.g1fb5d106.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-03-08T08:24:41.290 INFO:teuthology.orchestra.run.smithi144.stdout:(2/174): ceph-fuse-19.0.0-1670.g1fb5d106.el9.x8 2.2 MB/s | 887 kB 00:00 2024-03-08T08:24:41.374 INFO:teuthology.orchestra.run.smithi144.stdout:(3/174): ceph-immutable-object-cache-19.0.0-167 1.7 MB/s | 145 kB 00:00 2024-03-08T08:24:42.075 INFO:teuthology.orchestra.run.smithi144.stdout:(4/174): ceph-mds-19.0.0-1670.g1fb5d106.el9.x86 3.0 MB/s | 2.1 MB 00:00 2024-03-08T08:24:42.266 INFO:teuthology.orchestra.run.smithi144.stdout:(5/174): ceph-base-19.0.0-1670.g1fb5d106.el9.x8 3.4 MB/s | 5.3 MB 00:01 2024-03-08T08:24:42.499 INFO:teuthology.orchestra.run.smithi144.stdout:(6/174): ceph-mgr-19.0.0-1670.g1fb5d106.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-03-08T08:24:43.424 INFO:teuthology.orchestra.run.smithi144.stdout:(7/174): ceph-mon-19.0.0-1670.g1fb5d106.el9.x86 4.0 MB/s | 4.7 MB 00:01 2024-03-08T08:24:44.383 INFO:teuthology.orchestra.run.smithi144.stdout:(8/174): ceph-common-19.0.0-1670.g1fb5d106.el9. 5.4 MB/s | 20 MB 00:03 2024-03-08T08:24:44.466 INFO:teuthology.orchestra.run.smithi144.stdout:(9/174): ceph-selinux-19.0.0-1670.g1fb5d106.el9 303 kB/s | 25 kB 00:00 2024-03-08T08:24:46.158 INFO:teuthology.orchestra.run.smithi144.stdout:(10/174): ceph-radosgw-19.0.0-1670.g1fb5d106.el 3.6 MB/s | 9.8 MB 00:02 2024-03-08T08:24:46.250 INFO:teuthology.orchestra.run.smithi144.stdout:(11/174): cephfs-mirror-19.0.0-1670.g1fb5d106.e 2.4 MB/s | 220 kB 00:00 2024-03-08T08:24:46.334 INFO:teuthology.orchestra.run.smithi144.stdout:(12/174): cephfs-shell-19.0.0-1670.g1fb5d106.el 325 kB/s | 27 kB 00:00 2024-03-08T08:24:46.417 INFO:teuthology.orchestra.run.smithi144.stdout:(13/174): libcephfs-devel-19.0.0-1670.g1fb5d106 375 kB/s | 31 kB 00:00 2024-03-08T08:24:46.718 INFO:teuthology.orchestra.run.smithi144.stdout:(14/174): ceph-osd-19.0.0-1670.g1fb5d106.el9.x8 4.0 MB/s | 17 MB 00:04 2024-03-08T08:24:46.751 INFO:teuthology.orchestra.run.smithi144.stdout:(15/174): libcephfs2-19.0.0-1670.g1fb5d106.el9. 2.2 MB/s | 745 kB 00:00 2024-03-08T08:24:46.801 INFO:teuthology.orchestra.run.smithi144.stdout:(16/174): libcephsqlite-19.0.0-1670.g1fb5d106.e 1.9 MB/s | 163 kB 00:00 2024-03-08T08:24:46.835 INFO:teuthology.orchestra.run.smithi144.stdout:(17/174): librados-devel-19.0.0-1670.g1fb5d106. 1.5 MB/s | 127 kB 00:00 2024-03-08T08:24:46.969 INFO:teuthology.orchestra.run.smithi144.stdout:(18/174): libradosstriper1-19.0.0-1670.g1fb5d10 2.9 MB/s | 501 kB 00:00 2024-03-08T08:24:47.052 INFO:teuthology.orchestra.run.smithi144.stdout:(19/174): python3-ceph-argparse-19.0.0-1670.g1f 544 kB/s | 45 kB 00:00 2024-03-08T08:24:47.152 INFO:teuthology.orchestra.run.smithi144.stdout:(20/174): python3-ceph-common-19.0.0-1670.g1fb5 1.3 MB/s | 131 kB 00:00 2024-03-08T08:24:47.261 INFO:teuthology.orchestra.run.smithi144.stdout:(21/174): python3-cephfs-19.0.0-1670.g1fb5d106. 1.5 MB/s | 162 kB 00:00 2024-03-08T08:24:47.436 INFO:teuthology.orchestra.run.smithi144.stdout:(22/174): python3-rados-19.0.0-1670.g1fb5d106.e 1.8 MB/s | 320 kB 00:00 2024-03-08T08:24:47.537 INFO:teuthology.orchestra.run.smithi144.stdout:(23/174): python3-rbd-19.0.0-1670.g1fb5d106.el9 2.9 MB/s | 298 kB 00:00 2024-03-08T08:24:47.637 INFO:teuthology.orchestra.run.smithi144.stdout:(24/174): python3-rgw-19.0.0-1670.g1fb5d106.el9 995 kB/s | 99 kB 00:00 2024-03-08T08:24:47.712 INFO:teuthology.orchestra.run.smithi144.stdout:(25/174): rbd-fuse-19.0.0-1670.g1fb5d106.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-08T08:24:48.262 INFO:teuthology.orchestra.run.smithi144.stdout:(26/174): librgw2-19.0.0-1670.g1fb5d106.el9.x86 3.6 MB/s | 5.1 MB 00:01 2024-03-08T08:24:48.329 INFO:teuthology.orchestra.run.smithi144.stdout:(27/174): rbd-nbd-19.0.0-1670.g1fb5d106.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-08T08:24:48.396 INFO:teuthology.orchestra.run.smithi144.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1670.g 470 kB/s | 31 kB 00:00 2024-03-08T08:24:48.447 INFO:teuthology.orchestra.run.smithi144.stdout:(29/174): rbd-mirror-19.0.0-1670.g1fb5d106.el9. 4.2 MB/s | 3.1 MB 00:00 2024-03-08T08:24:48.472 INFO:teuthology.orchestra.run.smithi144.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1670.g1fb5d10 1.8 MB/s | 138 kB 00:00 2024-03-08T08:24:48.781 INFO:teuthology.orchestra.run.smithi144.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1670.g1fb5d 5.3 MB/s | 1.8 MB 00:00 2024-03-08T08:24:48.847 INFO:teuthology.orchestra.run.smithi144.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1670.g1f 3.6 MB/s | 248 kB 00:00 2024-03-08T08:24:48.915 INFO:teuthology.orchestra.run.smithi144.stdout:(33/174): ceph-mgr-rook-19.0.0-1670.g1fb5d106.e 740 kB/s | 49 kB 00:00 2024-03-08T08:24:48.981 INFO:teuthology.orchestra.run.smithi144.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1670.g1 230 kB/s | 15 kB 00:00 2024-03-08T08:24:49.065 INFO:teuthology.orchestra.run.smithi144.stdout:(35/174): ceph-volume-19.0.0-1670.g1fb5d106.el9 3.1 MB/s | 264 kB 00:00 2024-03-08T08:24:49.242 INFO:teuthology.orchestra.run.smithi144.stdout:(36/174): cephadm-19.0.0-1670.g1fb5d106.el9.noa 4.2 MB/s | 748 kB 00:00 2024-03-08T08:24:49.307 INFO:teuthology.orchestra.run.smithi144.stdout:(37/174): cephfs-top-19.0.0-1670.g1fb5d106.el9. 275 kB/s | 18 kB 00:00 2024-03-08T08:24:49.758 INFO:teuthology.orchestra.run.smithi144.stdout:(38/174): ceph-test-19.0.0-1670.g1fb5d106.el9.x 9.0 MB/s | 47 MB 00:05 2024-03-08T08:24:49.933 INFO:teuthology.orchestra.run.smithi144.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-03-08T08:24:50.116 INFO:teuthology.orchestra.run.smithi144.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 819 kB/s | 662 kB 00:00 2024-03-08T08:24:50.141 INFO:teuthology.orchestra.run.smithi144.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 578 kB/s | 106 kB 00:00 2024-03-08T08:24:50.192 INFO:teuthology.orchestra.run.smithi144.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-03-08T08:24:50.217 INFO:teuthology.orchestra.run.smithi144.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 421 kB/s | 32 kB 00:00 2024-03-08T08:24:50.242 INFO:teuthology.orchestra.run.smithi144.stdout:(44/174): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 920 kB 00:00 2024-03-08T08:24:50.309 INFO:teuthology.orchestra.run.smithi144.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 186 kB/s | 17 kB 00:00 2024-03-08T08:24:50.335 INFO:teuthology.orchestra.run.smithi144.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 110 kB/s | 10 kB 00:00 2024-03-08T08:24:50.401 INFO:teuthology.orchestra.run.smithi144.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-03-08T08:24:50.426 INFO:teuthology.orchestra.run.smithi144.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-03-08T08:24:50.452 INFO:teuthology.orchestra.run.smithi144.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 381 kB/s | 44 kB 00:00 2024-03-08T08:24:50.477 INFO:teuthology.orchestra.run.smithi144.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-08T08:24:50.510 INFO:teuthology.orchestra.run.smithi144.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 46 kB 00:00 2024-03-08T08:24:50.552 INFO:teuthology.orchestra.run.smithi144.stdout:(52/174): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 578 kB 00:00 2024-03-08T08:24:50.578 INFO:teuthology.orchestra.run.smithi144.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.7 MB/s | 282 kB 00:00 2024-03-08T08:24:50.603 INFO:teuthology.orchestra.run.smithi144.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-03-08T08:24:50.628 INFO:teuthology.orchestra.run.smithi144.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-08T08:24:50.662 INFO:teuthology.orchestra.run.smithi144.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 588 kB/s | 34 kB 00:00 2024-03-08T08:24:50.854 INFO:teuthology.orchestra.run.smithi144.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-03-08T08:24:50.904 INFO:teuthology.orchestra.run.smithi144.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 5.5 MB/s | 1.5 MB 00:00 2024-03-08T08:24:50.929 INFO:teuthology.orchestra.run.smithi144.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.0 MB/s | 811 kB 00:00 2024-03-08T08:24:50.955 INFO:teuthology.orchestra.run.smithi144.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-08T08:24:51.030 INFO:teuthology.orchestra.run.smithi144.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-03-08T08:24:51.188 INFO:teuthology.orchestra.run.smithi144.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 22 MB/s | 5.7 MB 00:00 2024-03-08T08:24:51.214 INFO:teuthology.orchestra.run.smithi144.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 300 kB/s | 78 kB 00:00 2024-03-08T08:24:51.339 INFO:teuthology.orchestra.run.smithi144.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-03-08T08:24:51.356 INFO:teuthology.orchestra.run.smithi144.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 148 kB/s | 24 kB 00:00 2024-03-08T08:24:51.373 INFO:teuthology.orchestra.run.smithi144.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-08T08:24:51.398 INFO:teuthology.orchestra.run.smithi144.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-08T08:24:51.423 INFO:teuthology.orchestra.run.smithi144.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 634 kB/s | 32 kB 00:00 2024-03-08T08:24:51.582 INFO:teuthology.orchestra.run.smithi144.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-03-08T08:24:51.607 INFO:teuthology.orchestra.run.smithi144.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-03-08T08:24:51.632 INFO:teuthology.orchestra.run.smithi144.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 842 kB/s | 174 kB 00:00 2024-03-08T08:24:51.657 INFO:teuthology.orchestra.run.smithi144.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 922 kB/s | 69 kB 00:00 2024-03-08T08:24:51.691 INFO:teuthology.orchestra.run.smithi144.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 706 kB/s | 41 kB 00:00 2024-03-08T08:24:51.717 INFO:teuthology.orchestra.run.smithi144.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-08T08:24:51.742 INFO:teuthology.orchestra.run.smithi144.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-08T08:24:51.767 INFO:teuthology.orchestra.run.smithi144.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-08T08:24:51.792 INFO:teuthology.orchestra.run.smithi144.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 629 kB/s | 47 kB 00:00 2024-03-08T08:24:51.818 INFO:teuthology.orchestra.run.smithi144.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-03-08T08:24:52.201 INFO:teuthology.orchestra.run.smithi144.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 94 kB/s | 39 kB 00:00 2024-03-08T08:24:52.235 INFO:teuthology.orchestra.run.smithi144.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 99 kB/s | 41 kB 00:00 2024-03-08T08:24:52.393 INFO:teuthology.orchestra.run.smithi144.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-03-08T08:24:52.418 INFO:teuthology.orchestra.run.smithi144.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-08T08:24:52.444 INFO:teuthology.orchestra.run.smithi144.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 274 kB/s | 56 kB 00:00 2024-03-08T08:24:52.469 INFO:teuthology.orchestra.run.smithi144.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 836 kB/s | 63 kB 00:00 2024-03-08T08:24:52.503 INFO:teuthology.orchestra.run.smithi144.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 757 kB/s | 44 kB 00:00 2024-03-08T08:24:52.520 INFO:teuthology.orchestra.run.smithi144.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-08T08:24:52.545 INFO:teuthology.orchestra.run.smithi144.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-08T08:24:52.595 INFO:teuthology.orchestra.run.smithi144.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-08T08:24:52.620 INFO:teuthology.orchestra.run.smithi144.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-08T08:24:52.646 INFO:teuthology.orchestra.run.smithi144.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-08T08:24:52.671 INFO:teuthology.orchestra.run.smithi144.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-08T08:24:52.938 INFO:teuthology.orchestra.run.smithi144.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 642 kB/s | 187 kB 00:00 2024-03-08T08:24:52.988 INFO:teuthology.orchestra.run.smithi144.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 616 kB/s | 195 kB 00:00 2024-03-08T08:24:53.005 INFO:teuthology.orchestra.run.smithi144.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-08T08:24:53.172 INFO:teuthology.orchestra.run.smithi144.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-03-08T08:24:53.264 INFO:teuthology.orchestra.run.smithi144.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 78 kB/s | 21 kB 00:00 2024-03-08T08:24:53.323 INFO:teuthology.orchestra.run.smithi144.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 80 kB/s | 25 kB 00:00 2024-03-08T08:24:53.456 INFO:teuthology.orchestra.run.smithi144.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-08T08:24:53.532 INFO:teuthology.orchestra.run.smithi144.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-03-08T08:24:53.557 INFO:teuthology.orchestra.run.smithi144.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.1 MB/s | 216 kB 00:00 2024-03-08T08:24:53.624 INFO:teuthology.orchestra.run.smithi144.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 380 kB/s | 35 kB 00:00 2024-03-08T08:24:53.657 INFO:teuthology.orchestra.run.smithi144.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 444 kB/s | 44 kB 00:00 2024-03-08T08:24:53.749 INFO:teuthology.orchestra.run.smithi144.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 749 kB/s | 94 kB 00:00 2024-03-08T08:24:53.850 INFO:teuthology.orchestra.run.smithi144.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 6.1 MB/s | 4.1 MB 00:00 2024-03-08T08:24:53.908 INFO:teuthology.orchestra.run.smithi144.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-03-08T08:24:53.950 INFO:teuthology.orchestra.run.smithi144.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-03-08T08:24:54.017 INFO:teuthology.orchestra.run.smithi144.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-03-08T08:24:54.159 INFO:teuthology.orchestra.run.smithi144.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-08T08:24:54.225 INFO:teuthology.orchestra.run.smithi144.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-03-08T08:24:54.293 INFO:teuthology.orchestra.run.smithi144.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 728 kB/s | 49 kB 00:00 2024-03-08T08:24:54.468 INFO:teuthology.orchestra.run.smithi144.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 385 kB/s | 67 kB 00:00 2024-03-08T08:24:54.593 INFO:teuthology.orchestra.run.smithi144.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-03-08T08:24:54.627 INFO:teuthology.orchestra.run.smithi144.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 953 kB/s | 151 kB 00:00 2024-03-08T08:24:54.719 INFO:teuthology.orchestra.run.smithi144.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-03-08T08:24:54.777 INFO:teuthology.orchestra.run.smithi144.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 747 kB/s | 43 kB 00:00 2024-03-08T08:24:54.836 INFO:teuthology.orchestra.run.smithi144.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-03-08T08:24:54.886 INFO:teuthology.orchestra.run.smithi144.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-03-08T08:24:54.961 INFO:teuthology.orchestra.run.smithi144.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-08T08:24:54.995 INFO:teuthology.orchestra.run.smithi144.stdout:(119/174): python3-asyncssh-2.13.2-4.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-03-08T08:24:55.053 INFO:teuthology.orchestra.run.smithi144.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-08T08:24:55.121 INFO:teuthology.orchestra.run.smithi144.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 528 kB/s | 35 kB 00:00 2024-03-08T08:24:55.179 INFO:teuthology.orchestra.run.smithi144.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-08T08:24:55.213 INFO:teuthology.orchestra.run.smithi144.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-03-08T08:24:55.355 INFO:teuthology.orchestra.run.smithi144.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-03-08T08:24:55.413 INFO:teuthology.orchestra.run.smithi144.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-03-08T08:24:55.472 INFO:teuthology.orchestra.run.smithi144.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 304 kB/s | 18 kB 00:00 2024-03-08T08:24:55.522 INFO:teuthology.orchestra.run.smithi144.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-03-08T08:24:55.548 INFO:teuthology.orchestra.run.smithi144.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-03-08T08:24:55.590 INFO:teuthology.orchestra.run.smithi144.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-03-08T08:24:55.619 INFO:teuthology.orchestra.run.smithi144.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-03-08T08:24:55.698 INFO:teuthology.orchestra.run.smithi144.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-03-08T08:24:55.832 INFO:teuthology.orchestra.run.smithi144.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-03-08T08:24:55.857 INFO:teuthology.orchestra.run.smithi144.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 497 kB/s | 79 kB 00:00 2024-03-08T08:24:55.916 INFO:teuthology.orchestra.run.smithi144.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-03-08T08:24:55.983 INFO:teuthology.orchestra.run.smithi144.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-03-08T08:24:56.050 INFO:teuthology.orchestra.run.smithi144.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-08T08:24:56.100 INFO:teuthology.orchestra.run.smithi144.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-03-08T08:24:56.134 INFO:teuthology.orchestra.run.smithi144.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-03-08T08:24:56.234 INFO:teuthology.orchestra.run.smithi144.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-08T08:24:56.259 INFO:teuthology.orchestra.run.smithi144.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 193 kB/s | 31 kB 00:00 2024-03-08T08:24:56.301 INFO:teuthology.orchestra.run.smithi144.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-03-08T08:24:56.327 INFO:teuthology.orchestra.run.smithi144.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-03-08T08:24:56.377 INFO:teuthology.orchestra.run.smithi144.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-08T08:24:56.552 INFO:teuthology.orchestra.run.smithi144.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 512 kB/s | 90 kB 00:00 2024-03-08T08:24:56.755 INFO:teuthology.orchestra.run.smithi144.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-03-08T08:24:56.777 INFO:teuthology.orchestra.run.smithi144.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 509 kB/s | 230 kB 00:00 2024-03-08T08:24:56.820 INFO:teuthology.orchestra.run.smithi144.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 312 kB/s | 20 kB 00:00 2024-03-08T08:24:57.086 INFO:teuthology.orchestra.run.smithi144.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 621 kB/s | 192 kB 00:00 2024-03-08T08:24:57.195 INFO:teuthology.orchestra.run.smithi144.stdout:(149/174): protobuf-compiler-3.14.0-13.el9.x86_ 251 kB/s | 863 kB 00:03 2024-03-08T08:24:57.279 INFO:teuthology.orchestra.run.smithi144.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-03-08T08:24:57.646 INFO:teuthology.orchestra.run.smithi144.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 943 kB/s | 321 kB 00:00 2024-03-08T08:24:57.929 INFO:teuthology.orchestra.run.smithi144.stdout:(152/174): librados2-19.0.0-1670.g1fb5d106.el9. 4.0 MB/s | 3.4 MB 00:00 2024-03-08T08:24:59.021 INFO:teuthology.orchestra.run.smithi144.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-03-08T08:24:59.105 INFO:teuthology.orchestra.run.smithi144.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 514 kB/s | 42 kB 00:00 2024-03-08T08:24:59.288 INFO:teuthology.orchestra.run.smithi144.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.3 MB/s | 248 kB 00:00 2024-03-08T08:24:59.364 INFO:teuthology.orchestra.run.smithi144.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 953 kB/s | 71 kB 00:00 2024-03-08T08:24:59.456 INFO:teuthology.orchestra.run.smithi144.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 398 kB/s | 36 kB 00:00 2024-03-08T08:24:59.614 INFO:teuthology.orchestra.run.smithi144.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.2 MB/s | 192 kB 00:00 2024-03-08T08:24:59.790 INFO:teuthology.orchestra.run.smithi144.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.4 MB/s | 256 kB 00:00 2024-03-08T08:24:59.865 INFO:teuthology.orchestra.run.smithi144.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 308 kB/s | 23 kB 00:00 2024-03-08T08:24:59.982 INFO:teuthology.orchestra.run.smithi144.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 929 kB/s | 108 kB 00:00 2024-03-08T08:25:00.124 INFO:teuthology.orchestra.run.smithi144.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-03-08T08:25:00.265 INFO:teuthology.orchestra.run.smithi144.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 959 kB/s | 136 kB 00:00 2024-03-08T08:25:00.341 INFO:teuthology.orchestra.run.smithi144.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 853 kB/s | 64 kB 00:00 2024-03-08T08:25:00.424 INFO:teuthology.orchestra.run.smithi144.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 344 kB/s | 29 kB 00:00 2024-03-08T08:25:00.816 INFO:teuthology.orchestra.run.smithi144.stdout:(166/174): NetworkManager-1.46.0-1.el9.x86_64.r 739 kB/s | 2.3 MB 00:03 2024-03-08T08:25:01.167 INFO:teuthology.orchestra.run.smithi144.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 1.6 MB/s | 1.2 MB 00:00 2024-03-08T08:25:03.399 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 703 kB/s | 21 MB 00:29 2024-03-08T08:25:03.483 INFO:teuthology.orchestra.run.smithi144.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 2.3 MB 00:02 2024-03-08T08:25:03.967 INFO:teuthology.orchestra.run.smithi144.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 965 kB/s | 465 kB 00:00 2024-03-08T08:25:04.042 INFO:teuthology.orchestra.run.smithi144.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 417 kB/s | 31 kB 00:00 2024-03-08T08:25:04.151 INFO:teuthology.orchestra.run.smithi144.stdout:(171/174): librbd1-19.0.0-1670.g1fb5d106.el9.x8 461 kB/s | 3.1 MB 00:06 2024-03-08T08:25:04.251 INFO:teuthology.orchestra.run.smithi144.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 643 kB/s | 2.2 MB 00:03 2024-03-08T08:25:04.443 INFO:teuthology.orchestra.run.smithi144.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 255 kB/s | 74 kB 00:00 2024-03-08T08:25:04.818 INFO:teuthology.orchestra.run.smithi144.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 1.1 MB 00:00 2024-03-08T08:25:04.822 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-08T08:25:04.822 INFO:teuthology.orchestra.run.smithi144.stdout:Total 9.0 MB/s | 223 MB 00:24 2024-03-08T08:25:05.286 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-08T08:25:05.598 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-08T08:25:05.599 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-08T08:25:07.676 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-08T08:25:07.676 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 10 kB/s | 1.7 kB 00:00 2024-03-08T08:25:10.984 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T08:25:10.984 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T08:25:10.997 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T08:25:10.998 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T08:25:10.998 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T08:25:10.999 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T08:25:11.001 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T08:25:11.004 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T08:25:11.005 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T08:25:11.084 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-08T08:25:11.101 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-08T08:25:11.101 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-03-08T08:25:11.101 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-08T08:25:11.102 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-03-08T08:25:11.102 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T08:25:11.102 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 6.6 k 2024-03-08T08:25:11.102 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.3 M 2024-03-08T08:25:11.102 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 887 k 2024-03-08T08:25:11.102 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 145 k 2024-03-08T08:25:11.103 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 1.5 M 2024-03-08T08:25:11.103 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 138 k 2024-03-08T08:25:11.103 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 1.8 M 2024-03-08T08:25:11.103 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 7.4 M 2024-03-08T08:25:11.103 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 49 k 2024-03-08T08:25:11.103 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 9.8 M 2024-03-08T08:25:11.104 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 47 M 2024-03-08T08:25:11.104 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 264 k 2024-03-08T08:25:11.104 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 748 k 2024-03-08T08:25:11.104 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 220 k 2024-03-08T08:25:11.104 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 27 k 2024-03-08T08:25:11.104 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 18 k 2024-03-08T08:25:11.105 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T08:25:11.105 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T08:25:11.105 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T08:25:11.105 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 31 k 2024-03-08T08:25:11.105 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 745 k 2024-03-08T08:25:11.105 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 127 k 2024-03-08T08:25:11.106 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T08:25:11.106 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T08:25:11.106 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T08:25:11.106 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T08:25:11.106 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T08:25:11.107 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T08:25:11.107 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 162 k 2024-03-08T08:25:11.107 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 320 k 2024-03-08T08:25:11.107 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 298 k 2024-03-08T08:25:11.107 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 99 k 2024-03-08T08:25:11.107 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 85 k 2024-03-08T08:25:11.108 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T08:25:11.108 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 170 k 2024-03-08T08:25:11.108 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T08:25:11.108 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-03-08T08:25:11.108 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T08:25:11.108 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T08:25:11.109 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T08:25:11.109 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T08:25:11.109 INFO:teuthology.orchestra.run.smithi105.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T08:25:11.109 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T08:25:11.109 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T08:25:11.109 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T08:25:11.110 INFO:teuthology.orchestra.run.smithi105.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T08:25:11.110 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T08:25:11.110 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T08:25:11.110 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T08:25:11.110 INFO:teuthology.orchestra.run.smithi105.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T08:25:11.110 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.4 M 2024-03-08T08:25:11.111 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 3.1 M 2024-03-08T08:25:11.111 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T08:25:11.111 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T08:25:11.111 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T08:25:11.111 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T08:25:11.111 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T08:25:11.112 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T08:25:11.112 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T08:25:11.112 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T08:25:11.112 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T08:25:11.112 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-03-08T08:25:11.112 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 20 M 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 31 k 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 2.1 M 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 248 k 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 4.7 M 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 17 M 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1670.g1fb5d106.el9 ceph-noarch 15 k 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 25 k 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T08:25:11.113 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T08:25:11.114 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T08:25:11.114 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T08:25:11.114 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T08:25:11.114 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T08:25:11.114 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T08:25:11.114 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T08:25:11.114 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T08:25:11.119 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 163 k 2024-03-08T08:25:11.119 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T08:25:11.120 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T08:25:11.120 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T08:25:11.120 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T08:25:11.120 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T08:25:11.120 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 501 k 2024-03-08T08:25:11.121 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T08:25:11.121 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 5.1 M 2024-03-08T08:25:11.121 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T08:25:11.121 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T08:25:11.121 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T08:25:11.121 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T08:25:11.122 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T08:25:11.122 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T08:25:11.122 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T08:25:11.122 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T08:25:11.122 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T08:25:11.122 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T08:25:11.122 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T08:25:11.123 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T08:25:11.123 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T08:25:11.123 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T08:25:11.123 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T08:25:11.123 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T08:25:11.123 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T08:25:11.124 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T08:25:11.124 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 45 k 2024-03-08T08:25:11.124 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-1670.g1fb5d106.el9 ceph 131 k 2024-03-08T08:25:11.124 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T08:25:11.124 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T08:25:11.124 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T08:25:11.125 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T08:25:11.125 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T08:25:11.125 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T08:25:11.125 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T08:25:11.125 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T08:25:11.125 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T08:25:11.126 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T08:25:11.126 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T08:25:11.126 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T08:25:11.126 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T08:25:11.126 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T08:25:11.127 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T08:25:11.127 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T08:25:11.127 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T08:25:11.127 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T08:25:11.127 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T08:25:11.127 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T08:25:11.127 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T08:25:11.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T08:25:11.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T08:25:11.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T08:25:11.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T08:25:11.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T08:25:11.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T08:25:11.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T08:25:11.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T08:25:11.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T08:25:11.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T08:25:11.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T08:25:11.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T08:25:11.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T08:25:11.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T08:25:11.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T08:25:11.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T08:25:11.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T08:25:11.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T08:25:11.131 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T08:25:11.131 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T08:25:11.131 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T08:25:11.131 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T08:25:11.131 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T08:25:11.131 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T08:25:11.131 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T08:25:11.132 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T08:25:11.132 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T08:25:11.132 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T08:25:11.132 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T08:25:11.132 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T08:25:11.132 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T08:25:11.133 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T08:25:11.133 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T08:25:11.133 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T08:25:11.133 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T08:25:11.133 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T08:25:11.133 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T08:25:11.134 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T08:25:11.134 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T08:25:11.134 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T08:25:11.134 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T08:25:11.134 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T08:25:11.134 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-03-08T08:25:11.135 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T08:25:11.135 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:25:11.135 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-08T08:25:11.135 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-08T08:25:11.135 INFO:teuthology.orchestra.run.smithi105.stdout:Install 150 Packages 2024-03-08T08:25:11.135 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 24 Packages 2024-03-08T08:25:11.136 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:25:11.136 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 223 M 2024-03-08T08:25:11.137 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi105.stdout:(1/174): ceph-19.0.0-1670.g1fb5d106.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-03-08T08:25:12.364 INFO:teuthology.orchestra.run.smithi105.stdout:(2/174): ceph-fuse-19.0.0-1670.g1fb5d106.el9.x8 3.2 MB/s | 887 kB 00:00 2024-03-08T08:25:12.431 INFO:teuthology.orchestra.run.smithi105.stdout:(3/174): ceph-immutable-object-cache-19.0.0-167 2.1 MB/s | 145 kB 00:00 2024-03-08T08:25:12.723 INFO:teuthology.orchestra.run.smithi105.stdout:(4/174): ceph-mds-19.0.0-1670.g1fb5d106.el9.x86 7.3 MB/s | 2.1 MB 00:00 2024-03-08T08:25:12.882 INFO:teuthology.orchestra.run.smithi105.stdout:(5/174): ceph-mgr-19.0.0-1670.g1fb5d106.el9.x86 9.4 MB/s | 1.5 MB 00:00 2024-03-08T08:25:13.232 INFO:teuthology.orchestra.run.smithi105.stdout:(6/174): ceph-mon-19.0.0-1670.g1fb5d106.el9.x86 13 MB/s | 4.7 MB 00:00 2024-03-08T08:25:14.240 INFO:teuthology.orchestra.run.smithi105.stdout:(7/174): ceph-osd-19.0.0-1670.g1fb5d106.el9.x86 17 MB/s | 17 MB 00:01 2024-03-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi105.stdout:(8/174): ceph-radosgw-19.0.0-1670.g1fb5d106.el9 18 MB/s | 9.8 MB 00:00 2024-03-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi105.stdout:(9/174): ceph-selinux-19.0.0-1670.g1fb5d106.el9 608 kB/s | 25 kB 00:00 2024-03-08T08:25:15.022 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-08T08:25:15.139 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T08:25:15.166 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T08:25:15.204 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T08:25:15.214 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T08:25:15.263 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T08:25:15.438 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T08:25:15.525 INFO:teuthology.orchestra.run.smithi105.stdout:(10/174): ceph-base-19.0.0-1670.g1fb5d106.el9.x 1.5 MB/s | 5.3 MB 00:03 2024-03-08T08:25:15.762 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T08:25:15.817 INFO:teuthology.orchestra.run.smithi105.stdout:(11/174): cephfs-mirror-19.0.0-1670.g1fb5d106.e 754 kB/s | 220 kB 00:00 2024-03-08T08:25:15.833 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi105.stdout:(12/174): cephfs-shell-19.0.0-1670.g1fb5d106.el 216 kB/s | 27 kB 00:00 2024-03-08T08:25:15.964 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T08:25:16.039 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T08:25:16.075 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T08:25:16.075 INFO:teuthology.orchestra.run.smithi105.stdout:(13/174): libcephfs-devel-19.0.0-1670.g1fb5d106 234 kB/s | 31 kB 00:00 2024-03-08T08:25:16.124 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T08:25:16.296 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T08:25:16.341 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T08:25:16.495 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T08:25:16.560 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T08:25:16.608 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T08:25:16.630 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T08:25:16.646 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T08:25:16.805 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T08:25:16.836 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T08:25:17.061 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T08:25:17.187 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T08:25:17.247 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T08:25:17.259 INFO:teuthology.orchestra.run.smithi105.stdout:(14/174): ceph-test-19.0.0-1670.g1fb5d106.el9.x 20 MB/s | 47 MB 00:02 2024-03-08T08:25:17.284 INFO:teuthology.orchestra.run.smithi105.stdout:(15/174): libcephfs2-19.0.0-1670.g1fb5d106.el9. 617 kB/s | 745 kB 00:01 2024-03-08T08:25:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:(16/174): libcephsqlite-19.0.0-1670.g1fb5d106.e 3.2 MB/s | 163 kB 00:00 2024-03-08T08:25:17.348 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T08:25:17.369 INFO:teuthology.orchestra.run.smithi105.stdout:(17/174): libradosstriper1-19.0.0-1670.g1fb5d10 8.4 MB/s | 501 kB 00:00 2024-03-08T08:25:17.410 INFO:teuthology.orchestra.run.smithi105.stdout:(18/174): librados-devel-19.0.0-1670.g1fb5d106. 1.0 MB/s | 127 kB 00:00 2024-03-08T08:25:17.426 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T08:25:17.527 INFO:teuthology.orchestra.run.smithi105.stdout:(19/174): python3-ceph-argparse-19.0.0-1670.g1f 387 kB/s | 45 kB 00:00 2024-03-08T08:25:17.658 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T08:25:17.678 INFO:teuthology.orchestra.run.smithi105.stdout:(20/174): librgw2-19.0.0-1670.g1fb5d106.el9.x86 17 MB/s | 5.1 MB 00:00 2024-03-08T08:25:17.736 INFO:teuthology.orchestra.run.smithi105.stdout:(21/174): python3-cephfs-19.0.0-1670.g1fb5d106. 2.7 MB/s | 162 kB 00:00 2024-03-08T08:25:17.762 INFO:teuthology.orchestra.run.smithi105.stdout:(22/174): python3-ceph-common-19.0.0-1670.g1fb5 561 kB/s | 131 kB 00:00 2024-03-08T08:25:17.782 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T08:25:17.787 INFO:teuthology.orchestra.run.smithi105.stdout:(23/174): python3-rados-19.0.0-1670.g1fb5d106.e 6.2 MB/s | 320 kB 00:00 2024-03-08T08:25:17.837 INFO:teuthology.orchestra.run.smithi105.stdout:(24/174): python3-rgw-19.0.0-1670.g1fb5d106.el9 1.9 MB/s | 99 kB 00:00 2024-03-08T08:25:17.886 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T08:25:17.887 INFO:teuthology.orchestra.run.smithi105.stdout:(25/174): rbd-fuse-19.0.0-1670.g1fb5d106.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-08T08:25:17.923 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T08:25:17.950 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T08:25:17.994 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T08:25:17.996 INFO:teuthology.orchestra.run.smithi105.stdout:(26/174): python3-rbd-19.0.0-1670.g1fb5d106.el9 1.2 MB/s | 298 kB 00:00 2024-03-08T08:25:18.068 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T08:25:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:(27/174): rbd-mirror-19.0.0-1670.g1fb5d106.el9. 16 MB/s | 3.1 MB 00:00 2024-03-08T08:25:18.113 INFO:teuthology.orchestra.run.smithi105.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1670.g 948 kB/s | 31 kB 00:00 2024-03-08T08:25:18.122 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T08:25:18.168 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T08:25:18.172 INFO:teuthology.orchestra.run.smithi105.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1670.g1fb5d10 2.3 MB/s | 138 kB 00:00 2024-03-08T08:25:18.206 INFO:teuthology.orchestra.run.smithi105.stdout:(30/174): rbd-nbd-19.0.0-1670.g1fb5d106.el9.x86 815 kB/s | 170 kB 00:00 2024-03-08T08:25:18.233 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T08:25:18.268 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T08:25:18.298 INFO:teuthology.orchestra.run.smithi105.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1670.g1fb5d 14 MB/s | 1.8 MB 00:00 2024-03-08T08:25:18.311 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T08:25:18.344 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T08:25:18.356 INFO:teuthology.orchestra.run.smithi105.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1670.g1f 4.2 MB/s | 248 kB 00:00 2024-03-08T08:25:18.372 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T08:25:18.398 INFO:teuthology.orchestra.run.smithi105.stdout:(33/174): ceph-mgr-rook-19.0.0-1670.g1fb5d106.e 1.2 MB/s | 49 kB 00:00 2024-03-08T08:25:18.440 INFO:teuthology.orchestra.run.smithi105.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1670.g1 369 kB/s | 15 kB 00:00 2024-03-08T08:25:18.440 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T08:25:18.476 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 39/198 2024-03-08T08:25:18.490 INFO:teuthology.orchestra.run.smithi105.stdout:(35/174): ceph-volume-19.0.0-1670.g1fb5d106.el9 5.2 MB/s | 264 kB 00:00 2024-03-08T08:25:18.521 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 40/198 2024-03-08T08:25:18.548 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 41/198 2024-03-08T08:25:18.557 INFO:teuthology.orchestra.run.smithi105.stdout:(36/174): cephadm-19.0.0-1670.g1fb5d106.el9.noa 11 MB/s | 748 kB 00:00 2024-03-08T08:25:18.599 INFO:teuthology.orchestra.run.smithi105.stdout:(37/174): cephfs-top-19.0.0-1670.g1fb5d106.el9. 439 kB/s | 18 kB 00:00 2024-03-08T08:25:18.626 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T08:25:18.680 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T08:25:18.754 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T08:25:18.896 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T08:25:18.949 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T08:25:19.008 INFO:teuthology.orchestra.run.smithi105.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-03-08T08:25:19.134 INFO:teuthology.orchestra.run.smithi105.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 7.2 MB/s | 920 kB 00:00 2024-03-08T08:25:19.200 INFO:teuthology.orchestra.run.smithi105.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-08T08:25:19.276 INFO:teuthology.orchestra.run.smithi105.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-03-08T08:25:19.334 INFO:teuthology.orchestra.run.smithi105.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 541 kB/s | 32 kB 00:00 2024-03-08T08:25:19.473 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T08:25:19.553 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T08:25:19.585 INFO:teuthology.orchestra.run.smithi105.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-03-08T08:25:19.633 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T08:25:19.652 INFO:teuthology.orchestra.run.smithi105.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 257 kB/s | 17 kB 00:00 2024-03-08T08:25:19.672 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T08:25:19.681 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T08:25:19.711 INFO:teuthology.orchestra.run.smithi105.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 172 kB/s | 10 kB 00:00 2024-03-08T08:25:19.716 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T08:25:19.762 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T08:25:19.777 INFO:teuthology.orchestra.run.smithi105.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-03-08T08:25:19.800 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 53/198 2024-03-08T08:25:19.830 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T08:25:19.836 INFO:teuthology.orchestra.run.smithi105.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 756 kB/s | 44 kB 00:00 2024-03-08T08:25:19.920 INFO:teuthology.orchestra.run.smithi105.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-03-08T08:25:19.995 INFO:teuthology.orchestra.run.smithi105.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 578 kB 00:00 2024-03-08T08:25:20.045 INFO:teuthology.orchestra.run.smithi105.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 925 kB/s | 46 kB 00:00 2024-03-08T08:25:20.101 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T08:25:20.121 INFO:teuthology.orchestra.run.smithi105.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-08T08:25:20.196 INFO:teuthology.orchestra.run.smithi105.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-08T08:25:20.272 INFO:teuthology.orchestra.run.smithi105.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-08T08:25:20.308 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T08:25:20.330 INFO:teuthology.orchestra.run.smithi105.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 588 kB/s | 34 kB 00:00 2024-03-08T08:25:20.390 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T08:25:20.435 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T08:25:20.575 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T08:25:20.580 INFO:teuthology.orchestra.run.smithi105.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-03-08T08:25:20.621 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T08:25:20.677 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T08:25:20.681 INFO:teuthology.orchestra.run.smithi105.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 15 MB/s | 1.5 MB 00:00 2024-03-08T08:25:20.748 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T08:25:20.764 INFO:teuthology.orchestra.run.smithi105.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 9.5 MB/s | 811 kB 00:00 2024-03-08T08:25:20.788 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T08:25:20.823 INFO:teuthology.orchestra.run.smithi105.stdout:(58/174): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2024-03-08T08:25:20.824 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T08:25:20.871 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T08:25:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T08:25:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T08:25:21.001 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T08:25:21.057 INFO:teuthology.orchestra.run.smithi105.stdout:(59/174): postgresql-server-13.11-1.el9.x86_64. 24 MB/s | 5.7 MB 00:00 2024-03-08T08:25:21.124 INFO:teuthology.orchestra.run.smithi105.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-08T08:25:21.143 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T08:25:21.174 INFO:teuthology.orchestra.run.smithi105.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 78 kB 00:00 2024-03-08T08:25:21.182 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T08:25:21.212 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T08:25:21.253 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi105.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-08T08:25:21.427 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T08:25:21.441 INFO:teuthology.orchestra.run.smithi105.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 491 kB/s | 24 kB 00:00 2024-03-08T08:25:21.469 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T08:25:21.491 INFO:teuthology.orchestra.run.smithi105.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-03-08T08:25:21.494 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T08:25:21.550 INFO:teuthology.orchestra.run.smithi105.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-08T08:25:21.580 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T08:25:21.601 INFO:teuthology.orchestra.run.smithi105.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-08T08:25:21.703 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T08:25:21.784 INFO:teuthology.orchestra.run.smithi105.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-08T08:25:21.811 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T08:25:21.843 INFO:teuthology.orchestra.run.smithi105.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 217 kB 00:00 2024-03-08T08:25:21.861 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T08:25:21.885 INFO:teuthology.orchestra.run.smithi105.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.1 MB/s | 174 kB 00:00 2024-03-08T08:25:21.910 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T08:25:21.935 INFO:teuthology.orchestra.run.smithi105.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-08T08:25:21.966 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T08:25:21.977 INFO:teuthology.orchestra.run.smithi105.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 988 kB/s | 41 kB 00:00 2024-03-08T08:25:22.036 INFO:teuthology.orchestra.run.smithi105.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-08T08:25:22.086 INFO:teuthology.orchestra.run.smithi105.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-08T08:25:22.137 INFO:teuthology.orchestra.run.smithi105.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-08T08:25:22.150 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T08:25:22.178 INFO:teuthology.orchestra.run.smithi105.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-08T08:25:22.198 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T08:25:22.212 INFO:teuthology.orchestra.run.smithi105.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-08T08:25:22.327 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T08:25:22.364 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T08:25:22.395 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T08:25:22.426 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T08:25:22.454 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T08:25:22.488 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T08:25:22.528 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T08:25:22.559 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T08:25:22.590 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T08:25:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T08:25:22.679 INFO:teuthology.orchestra.run.smithi105.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-03-08T08:25:22.700 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T08:25:22.712 INFO:teuthology.orchestra.run.smithi105.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-08T08:25:22.733 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T08:25:22.754 INFO:teuthology.orchestra.run.smithi105.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 984 kB/s | 41 kB 00:00 2024-03-08T08:25:22.775 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T08:25:22.809 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T08:25:22.813 INFO:teuthology.orchestra.run.smithi105.stdout:(80/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-08T08:25:22.864 INFO:teuthology.orchestra.run.smithi105.stdout:(81/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-08T08:25:22.898 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T08:25:22.922 INFO:teuthology.orchestra.run.smithi105.stdout:(82/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-03-08T08:25:22.981 INFO:teuthology.orchestra.run.smithi105.stdout:(83/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-03-08T08:25:22.981 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T08:25:23.031 INFO:teuthology.orchestra.run.smithi105.stdout:(84/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-08T08:25:23.032 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T08:25:23.064 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T08:25:23.090 INFO:teuthology.orchestra.run.smithi105.stdout:(85/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-08T08:25:23.110 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T08:25:23.182 INFO:teuthology.orchestra.run.smithi105.stdout:(86/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-08T08:25:23.208 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T08:25:23.249 INFO:teuthology.orchestra.run.smithi105.stdout:(87/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-08T08:25:23.257 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T08:25:23.316 INFO:teuthology.orchestra.run.smithi105.stdout:(88/174): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 124 kB 00:00 2024-03-08T08:25:23.374 INFO:teuthology.orchestra.run.smithi105.stdout:(89/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-03-08T08:25:23.741 INFO:teuthology.orchestra.run.smithi105.stdout:(90/174): jq-1.6-16.el9.x86_64.rpm 510 kB/s | 187 kB 00:00 2024-03-08T08:25:23.892 INFO:teuthology.orchestra.run.smithi105.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-03-08T08:25:23.990 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T08:25:23.992 INFO:teuthology.orchestra.run.smithi105.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-03-08T08:25:24.037 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T08:25:24.071 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T08:25:24.084 INFO:teuthology.orchestra.run.smithi105.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-03-08T08:25:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T08:25:24.209 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T08:25:24.234 INFO:teuthology.orchestra.run.smithi105.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 143 kB/s | 21 kB 00:00 2024-03-08T08:25:24.247 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T08:25:24.310 INFO:teuthology.orchestra.run.smithi105.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-03-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T08:25:24.486 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T08:25:24.567 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T08:25:24.861 INFO:teuthology.orchestra.run.smithi105.stdout:(96/174): ceph-mgr-diskprediction-local-19.0.0- 1.1 MB/s | 7.4 MB 00:06 2024-03-08T08:25:25.206 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T08:25:25.235 INFO:teuthology.orchestra.run.smithi105.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 655 kB/s | 245 kB 00:00 2024-03-08T08:25:25.312 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T08:25:25.402 INFO:teuthology.orchestra.run.smithi105.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-03-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi105.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-08T08:25:25.586 INFO:teuthology.orchestra.run.smithi105.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 381 kB/s | 35 kB 00:00 2024-03-08T08:25:25.653 INFO:teuthology.orchestra.run.smithi105.stdout:(101/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.1 MB/s | 4.1 MB 00:01 2024-03-08T08:25:25.670 INFO:teuthology.orchestra.run.smithi105.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 533 kB/s | 44 kB 00:00 2024-03-08T08:25:25.737 INFO:teuthology.orchestra.run.smithi105.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-08T08:25:25.762 INFO:teuthology.orchestra.run.smithi105.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-03-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi105.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-03-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi105.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 508 kB/s | 309 kB 00:00 2024-03-08T08:25:26.580 INFO:teuthology.orchestra.run.smithi105.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 671 kB/s | 548 kB 00:00 2024-03-08T08:25:26.622 INFO:teuthology.orchestra.run.smithi105.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-03-08T08:25:26.623 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T08:25:26.654 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T08:25:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T08:25:26.689 INFO:teuthology.orchestra.run.smithi105.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-03-08T08:25:26.712 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T08:25:26.781 INFO:teuthology.orchestra.run.smithi105.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-03-08T08:25:26.906 INFO:teuthology.orchestra.run.smithi105.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 539 kB/s | 67 kB 00:00 2024-03-08T08:25:27.020 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T08:25:27.051 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T08:25:27.081 INFO:teuthology.orchestra.run.smithi105.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 863 kB/s | 151 kB 00:00 2024-03-08T08:25:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T08:25:27.172 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 125/198 2024-03-08T08:25:27.208 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T08:25:27.683 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T08:25:27.713 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T08:25:27.775 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T08:25:27.806 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 129/198 2024-03-08T08:25:27.840 INFO:teuthology.orchestra.run.smithi105.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.1 MB/s | 838 kB 00:00 2024-03-08T08:25:28.407 INFO:teuthology.orchestra.run.smithi105.stdout:(114/174): ceph-common-19.0.0-1670.g1fb5d106.el 1.2 MB/s | 20 MB 00:16 2024-03-08T08:25:28.524 INFO:teuthology.orchestra.run.smithi105.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 801 kB/s | 548 kB 00:00 2024-03-08T08:25:28.582 INFO:teuthology.orchestra.run.smithi105.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 562 kB/s | 32 kB 00:00 2024-03-08T08:25:28.732 INFO:teuthology.orchestra.run.smithi105.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 133 kB/s | 43 kB 00:00 2024-03-08T08:25:28.974 INFO:teuthology.orchestra.run.smithi105.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 38 kB/s | 15 kB 00:00 2024-03-08T08:25:29.400 INFO:teuthology.orchestra.run.smithi105.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 843 kB/s | 358 kB 00:00 2024-03-08T08:25:29.475 INFO:teuthology.orchestra.run.smithi105.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 231 kB/s | 172 kB 00:00 2024-03-08T08:25:29.659 INFO:teuthology.orchestra.run.smithi105.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 193 kB/s | 35 kB 00:00 2024-03-08T08:25:29.762 INFO:teuthology.orchestra.run.smithi105.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 845 kB/s | 303 kB 00:00 2024-03-08T08:25:29.955 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T08:25:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T08:25:30.367 INFO:teuthology.orchestra.run.smithi105.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 286 kB/s | 203 kB 00:00 2024-03-08T08:25:30.551 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T08:25:30.589 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T08:25:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T08:25:30.679 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T08:25:30.679 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T08:25:30.679 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:30.715 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T08:25:30.818 INFO:teuthology.orchestra.run.smithi105.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 321 kB/s | 144 kB 00:00 2024-03-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi105.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-08T08:25:30.994 INFO:teuthology.orchestra.run.smithi105.stdout:(126/174): libarrow-9.0.0-10.el9.x86_64.rpm 1.0 MB/s | 4.4 MB 00:04 2024-03-08T08:25:31.010 INFO:teuthology.orchestra.run.smithi105.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-03-08T08:25:31.061 INFO:teuthology.orchestra.run.smithi105.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-03-08T08:25:31.127 INFO:teuthology.orchestra.run.smithi105.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-03-08T08:25:31.144 INFO:teuthology.orchestra.run.smithi105.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-03-08T08:25:31.220 INFO:teuthology.orchestra.run.smithi105.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 618 kB/s | 46 kB 00:00 2024-03-08T08:25:31.404 INFO:teuthology.orchestra.run.smithi105.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 430 kB/s | 79 kB 00:00 2024-03-08T08:25:31.704 INFO:teuthology.orchestra.run.smithi105.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 192 kB/s | 58 kB 00:00 2024-03-08T08:25:32.379 INFO:teuthology.orchestra.run.smithi105.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 403 kB/s | 272 kB 00:00 2024-03-08T08:25:32.429 INFO:teuthology.orchestra.run.smithi105.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-03-08T08:25:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 539 kB/s | 90 kB 00:00 2024-03-08T08:25:32.671 INFO:teuthology.orchestra.run.smithi105.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 377 kB/s | 28 kB 00:00 2024-03-08T08:25:32.713 INFO:teuthology.orchestra.run.smithi105.stdout:(138/174): python3-grpcio-1.46.7-10.el9.x86_64. 706 kB/s | 2.0 MB 00:02 2024-03-08T08:25:32.763 INFO:teuthology.orchestra.run.smithi105.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-03-08T08:25:32.914 INFO:teuthology.orchestra.run.smithi105.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 394 kB/s | 59 kB 00:00 2024-03-08T08:25:32.997 INFO:teuthology.orchestra.run.smithi105.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2024-03-08T08:25:33.098 INFO:teuthology.orchestra.run.smithi105.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 491 kB/s | 188 kB 00:00 2024-03-08T08:25:33.123 INFO:teuthology.orchestra.run.smithi105.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 407 kB/s | 51 kB 00:00 2024-03-08T08:25:33.265 INFO:teuthology.orchestra.run.smithi105.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 633 kB/s | 90 kB 00:00 2024-03-08T08:25:33.449 INFO:teuthology.orchestra.run.smithi105.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 655 kB/s | 230 kB 00:00 2024-03-08T08:25:33.499 INFO:teuthology.orchestra.run.smithi105.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-03-08T08:25:33.799 INFO:teuthology.orchestra.run.smithi105.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 638 kB/s | 192 kB 00:00 2024-03-08T08:25:33.908 INFO:teuthology.orchestra.run.smithi105.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 664 kB/s | 427 kB 00:00 2024-03-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi105.stdout:(149/174): librados2-19.0.0-1670.g1fb5d106.el9. 12 MB/s | 3.4 MB 00:00 2024-03-08T08:25:34.359 INFO:teuthology.orchestra.run.smithi105.stdout:(150/174): python3-kubernetes-26.1.0-2.el9.noar 324 kB/s | 1.0 MB 00:03 2024-03-08T08:25:34.409 INFO:teuthology.orchestra.run.smithi105.stdout:(151/174): librbd1-19.0.0-1670.g1fb5d106.el9.x8 15 MB/s | 3.1 MB 00:00 2024-03-08T08:25:34.442 INFO:teuthology.orchestra.run.smithi105.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-03-08T08:25:34.576 INFO:teuthology.orchestra.run.smithi105.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 14 MB/s | 1.8 MB 00:00 2024-03-08T08:25:34.676 INFO:teuthology.orchestra.run.smithi105.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 424 kB/s | 42 kB 00:00 2024-03-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi105.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 2.6 MB/s | 248 kB 00:00 2024-03-08T08:25:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-08T08:25:34.878 INFO:teuthology.orchestra.run.smithi105.stdout:(157/174): NetworkManager-1.46.0-1.el9.x86_64.r 4.9 MB/s | 2.3 MB 00:00 2024-03-08T08:25:34.952 INFO:teuthology.orchestra.run.smithi105.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-08T08:25:34.978 INFO:teuthology.orchestra.run.smithi105.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 256 kB/s | 36 kB 00:00 2024-03-08T08:25:35.045 INFO:teuthology.orchestra.run.smithi105.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 346 kB/s | 23 kB 00:00 2024-03-08T08:25:35.070 INFO:teuthology.orchestra.run.smithi105.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-08T08:25:35.137 INFO:teuthology.orchestra.run.smithi105.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 108 kB 00:00 2024-03-08T08:25:35.162 INFO:teuthology.orchestra.run.smithi105.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 155 kB 00:00 2024-03-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi105.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi105.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 849 kB/s | 64 kB 00:00 2024-03-08T08:25:35.279 INFO:teuthology.orchestra.run.smithi105.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 692 kB/s | 29 kB 00:00 2024-03-08T08:25:35.347 INFO:teuthology.orchestra.run.smithi105.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-03-08T08:25:35.480 INFO:teuthology.orchestra.run.smithi105.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-03-08T08:25:35.581 INFO:teuthology.orchestra.run.smithi105.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.8 MB/s | 2.3 MB 00:00 2024-03-08T08:25:35.648 INFO:teuthology.orchestra.run.smithi105.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.7 MB/s | 465 kB 00:00 2024-03-08T08:25:35.673 INFO:teuthology.orchestra.run.smithi105.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 339 kB/s | 31 kB 00:00 2024-03-08T08:25:35.798 INFO:teuthology.orchestra.run.smithi105.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 596 kB/s | 74 kB 00:00 2024-03-08T08:25:35.890 INFO:teuthology.orchestra.run.smithi105.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.4 MB/s | 1.1 MB 00:00 2024-03-08T08:25:36.898 INFO:teuthology.orchestra.run.smithi105.stdout:(174/174): thrift-0.15.0-2.el9.x86_64.rpm 523 kB/s | 1.6 MB 00:03 2024-03-08T08:25:36.905 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-08T08:25:36.905 INFO:teuthology.orchestra.run.smithi105.stdout:Total 8.6 MB/s | 223 MB 00:25 2024-03-08T08:25:37.353 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-08T08:25:37.661 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-08T08:25:37.661 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-08T08:25:39.757 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-08T08:25:39.758 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-08T08:25:42.388 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T08:25:42.388 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-03-08T08:25:42.388 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-03-08T08:25:42.388 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-03-08T08:25:42.388 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-03-08T08:25:42.388 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-03-08T08:25:42.388 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-03-08T08:25:42.389 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-03-08T08:25:42.389 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:42.554 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T08:25:42.569 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T08:25:42.569 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:42.569 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T08:25:42.569 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:44.057 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T08:25:44.083 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T08:25:44.083 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:44.084 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T08:25:44.084 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T08:25:44.084 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T08:25:44.084 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:44.758 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T08:25:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T08:25:44.781 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:44.781 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T08:25:44.781 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T08:25:44.781 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T08:25:44.781 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:44.876 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T08:25:45.203 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T08:25:45.233 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T08:25:45.264 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 139/198 2024-03-08T08:25:45.325 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 140/198 2024-03-08T08:25:45.348 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T08:25:45.574 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T08:25:45.607 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T08:25:46.572 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T08:25:46.599 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T08:25:46.700 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T08:25:46.745 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-08T08:25:46.850 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T08:25:46.862 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 144/198 2024-03-08T08:25:46.876 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T08:25:46.893 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T08:25:46.914 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T08:25:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T08:25:46.916 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:46.916 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T08:25:46.916 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T08:25:46.916 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T08:25:46.916 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:46.923 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T08:25:46.961 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T08:25:46.967 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T08:25:46.972 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T08:25:47.025 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 147/198 2024-03-08T08:25:47.029 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T08:25:47.084 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T08:25:47.497 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T08:25:47.561 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T08:25:47.737 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T08:25:47.823 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T08:25:47.856 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T08:25:47.882 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T08:25:47.901 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T08:25:47.912 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T08:25:47.935 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T08:25:47.935 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:47.936 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T08:25:47.936 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T08:25:47.936 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T08:25:47.936 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:47.980 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T08:25:48.002 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T08:25:48.002 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:48.002 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T08:25:48.002 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:48.047 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T08:25:48.278 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T08:25:48.301 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T08:25:48.367 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 17/198 2024-03-08T08:25:48.381 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T08:25:48.404 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T08:25:48.404 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:25:48.404 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T08:25:48.404 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T08:25:48.404 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T08:25:48.405 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:25:48.415 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T08:25:48.438 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T08:25:48.453 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T08:25:48.596 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T08:25:48.627 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T08:25:48.684 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T08:25:48.986 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T08:25:49.046 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T08:25:49.164 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T08:25:49.214 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T08:25:49.418 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T08:25:49.526 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T08:25:49.622 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T08:25:49.652 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T08:25:49.679 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T08:25:49.722 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T08:25:49.805 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T08:25:49.859 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T08:25:49.897 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T08:25:49.963 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T08:25:49.999 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T08:25:50.042 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T08:25:50.075 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T08:25:50.103 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T08:25:50.172 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T08:25:50.208 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 39/198 2024-03-08T08:25:50.251 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 40/198 2024-03-08T08:25:50.280 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 41/198 2024-03-08T08:25:50.358 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T08:25:50.419 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T08:25:50.493 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T08:25:50.628 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T08:25:50.680 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T08:25:51.195 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T08:25:51.275 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T08:25:51.363 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T08:25:51.404 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T08:25:51.412 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T08:25:51.445 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T08:25:51.494 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T08:25:51.540 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 53/198 2024-03-08T08:25:51.570 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T08:25:51.820 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T08:25:52.008 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T08:25:52.113 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T08:25:52.157 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T08:25:52.297 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T08:25:52.344 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T08:25:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T08:25:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T08:25:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T08:25:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T08:25:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T08:25:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T08:25:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T08:25:52.819 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T08:25:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T08:25:52.868 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 154/198 2024-03-08T08:25:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T08:25:52.898 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 155/198 2024-03-08T08:25:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T08:25:52.926 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T08:25:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T08:25:53.113 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 157/198 2024-03-08T08:25:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T08:25:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T08:25:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T08:25:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T08:25:53.233 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T08:25:53.285 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 159/198 2024-03-08T08:25:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T08:25:53.336 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T08:25:53.407 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T08:25:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T08:25:53.442 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T08:25:53.475 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 163/198 2024-03-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T08:25:53.602 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T08:25:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T08:25:53.671 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T08:25:53.676 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T08:25:53.718 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T08:25:53.872 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T08:25:53.922 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T08:25:54.066 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T08:25:54.112 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T08:25:54.144 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T08:25:54.174 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T08:25:54.202 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T08:25:54.237 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T08:25:54.276 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T08:25:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T08:25:54.338 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T08:25:54.348 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T08:25:54.385 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T08:25:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T08:25:54.413 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T08:25:54.457 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T08:25:54.507 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T08:25:54.548 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T08:25:54.583 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T08:25:54.671 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T08:25:54.753 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T08:25:54.797 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T08:25:54.829 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T08:25:54.876 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T08:25:54.974 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T08:25:55.032 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T08:25:55.210 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 170/198 2024-03-08T08:25:55.289 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T08:25:55.330 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T08:25:55.413 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T08:25:55.443 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T08:25:55.467 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T08:25:55.493 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T08:25:55.536 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T08:25:55.536 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T08:25:55.649 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T08:25:55.679 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T08:25:55.679 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T08:25:55.756 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T08:25:55.764 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T08:25:55.790 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T08:25:55.804 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T08:25:55.823 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T08:25:55.838 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T08:25:55.848 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T08:25:55.873 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T08:25:55.886 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T08:25:55.907 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T08:25:55.945 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T08:25:55.964 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T08:25:55.978 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T08:25:56.003 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T08:25:56.005 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T08:25:56.028 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T08:25:56.053 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T08:25:56.079 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T08:25:56.085 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T08:25:56.085 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T08:25:56.181 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T08:25:56.211 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T08:25:56.282 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T08:25:56.282 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T08:25:56.289 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T08:25:56.357 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T08:25:56.381 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T08:25:56.406 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T08:25:56.440 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T08:25:56.466 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T08:25:56.933 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T08:25:57.050 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T08:25:58.379 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T08:25:58.410 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T08:25:58.442 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T08:25:58.469 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T08:25:58.777 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T08:25:58.807 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T08:25:58.898 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 124/198 2024-03-08T08:25:58.929 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 125/198 2024-03-08T08:25:58.966 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T08:25:59.419 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T08:25:59.452 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T08:25:59.506 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 128/198 2024-03-08T08:25:59.537 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 129/198 2024-03-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 1/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 3/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 4/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 5/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 6/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 7/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 8/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 9/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 13/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 14/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 15/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 17/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 18/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 19/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 20/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 21/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 22/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 23/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 24/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 25/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 26/198 2024-03-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 27/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 28/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 29/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 30/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 31/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 32/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 33/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 34/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 35/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 36/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 37/198 2024-03-08T08:26:01.378 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 38/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 39/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T08:26:01.379 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T08:26:01.380 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T08:26:01.381 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T08:26:01.382 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T08:26:01.383 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T08:26:01.384 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T08:26:01.385 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T08:26:01.386 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T08:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T08:26:01.388 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T08:26:01.389 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T08:26:01.715 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T08:26:01.759 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T08:26:02.301 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 130/198 2024-03-08T08:26:02.337 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 131/198 2024-03-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T08:26:04.465 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T08:26:04.465 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:04.465 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-03-08T08:26:04.465 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.466 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T08:26:04.467 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.468 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.469 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-16.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T08:26:04.470 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.471 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T08:26:04.472 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T08:26:04.473 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T08:26:04.474 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T08:26:04.475 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T08:26:04.476 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.477 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T08:26:04.478 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T08:26:04.479 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T08:26:04.480 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T08:26:04.481 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:04.481 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-08T08:26:04.922 DEBUG:teuthology.parallel:result is None 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 132/198 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-03-08T08:26:14.214 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:14.401 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T08:26:14.417 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 133/198 2024-03-08T08:26:14.417 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:14.417 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T08:26:14.417 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:15.913 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T08:26:15.939 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 134/198 2024-03-08T08:26:15.939 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:15.939 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T08:26:15.939 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T08:26:15.939 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T08:26:15.939 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 135/198 2024-03-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:16.607 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 136/198 2024-03-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:16.733 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T08:26:17.092 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T08:26:17.123 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 138/198 2024-03-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 139/198 2024-03-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 140/198 2024-03-08T08:26:17.238 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T08:26:17.465 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 141/198 2024-03-08T08:26:17.506 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T08:26:18.462 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 142/198 2024-03-08T08:26:18.489 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T08:26:18.589 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 143/198 2024-03-08T08:26:18.752 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 144/198 2024-03-08T08:26:18.783 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T08:26:18.806 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 145/198 2024-03-08T08:26:18.806 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:18.806 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T08:26:18.806 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T08:26:18.806 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T08:26:18.807 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:18.851 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T08:26:18.863 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 146/198 2024-03-08T08:26:18.912 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 147/198 2024-03-08T08:26:19.770 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 149/198 2024-03-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:19.879 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T08:26:19.900 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 150/198 2024-03-08T08:26:19.901 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:19.901 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T08:26:19.901 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:19.954 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T08:26:19.976 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T08:26:19.976 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:19.976 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T08:26:19.976 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T08:26:19.976 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T08:26:19.976 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:20.289 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T08:26:20.310 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 152/198 2024-03-08T08:26:20.310 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T08:26:20.310 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T08:26:20.311 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T08:26:20.311 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T08:26:20.311 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:24.735 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T08:26:24.776 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 154/198 2024-03-08T08:26:24.806 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 155/198 2024-03-08T08:26:24.833 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T08:26:25.020 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 157/198 2024-03-08T08:26:25.149 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T08:26:25.206 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 159/198 2024-03-08T08:26:25.265 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T08:26:25.323 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T08:26:25.367 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T08:26:25.408 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 163/198 2024-03-08T08:26:25.448 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T08:26:25.478 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T08:26:25.607 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T08:26:25.653 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T08:26:26.302 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T08:26:26.342 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T08:26:26.362 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T08:26:27.173 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 170/198 2024-03-08T08:26:27.255 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T08:26:27.296 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T08:26:27.377 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T08:26:27.408 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T08:26:27.433 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T08:26:27.459 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T08:26:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T08:26:27.485 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T08:26:27.565 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T08:26:27.594 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T08:26:27.594 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T08:26:27.696 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T08:26:27.723 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T08:26:27.755 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T08:26:27.781 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T08:26:27.806 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T08:26:27.840 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T08:26:27.878 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T08:26:27.910 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T08:26:27.936 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T08:26:27.969 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T08:26:27.994 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T08:26:28.020 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T08:26:28.026 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T08:26:28.026 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T08:26:28.215 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T08:26:28.215 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T08:26:28.298 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T08:26:28.322 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T08:26:28.348 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T08:26:28.380 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T08:26:28.407 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T08:26:28.481 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T08:26:33.326 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T08:26:33.326 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 1/198 2024-03-08T08:26:33.326 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2/198 2024-03-08T08:26:33.329 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 3/198 2024-03-08T08:26:33.329 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 4/198 2024-03-08T08:26:33.329 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d 5/198 2024-03-08T08:26:33.329 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 6/198 2024-03-08T08:26:33.329 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 7/198 2024-03-08T08:26:33.329 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 8/198 2024-03-08T08:26:33.329 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 9/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 10/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 11/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 12/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 13/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 14/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_ 15/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 16/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 17/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_6 18/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86 19/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 20/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el 21/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9. 22/198 2024-03-08T08:26:33.330 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_6 23/198 2024-03-08T08:26:33.331 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 24/198 2024-03-08T08:26:33.331 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 25/198 2024-03-08T08:26:33.331 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 26/198 2024-03-08T08:26:33.331 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 27/198 2024-03-08T08:26:33.331 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 28/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 29/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106. 30/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noa 31/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.n 32/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb 33/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el 34/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 35/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.e 36/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 37/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 38/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 39/198 2024-03-08T08:26:33.346 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T08:26:33.347 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T08:26:33.348 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T08:26:33.349 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T08:26:33.350 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T08:26:33.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T08:26:33.352 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T08:26:33.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 151/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 153/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T08:26:33.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T08:26:33.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T08:26:33.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T08:26:33.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T08:26:33.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T08:26:33.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T08:26:36.310 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T08:26:36.311 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T08:26:36.312 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.313 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.314 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-1670.g1fb5d106.el9.noarch 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-16.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.315 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T08:26:36.316 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T08:26:36.317 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T08:26:36.318 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T08:26:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T08:26:36.320 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T08:26:36.322 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-1670.g1fb5d106.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T08:26:36.323 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:36.324 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-08T08:26:36.794 DEBUG:teuthology.parallel:result is None 2024-03-08T08:26:36.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T08:26:37.145 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T08:26:37.167 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-1670.g1fb5d106.el9 2024-03-08T08:26:37.167 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1670.g1fb5d106.el9 2024-03-08T08:26:37.168 INFO:teuthology.task.install:The correct ceph version 19.0.0-1670.g1fb5d106 is installed. 2024-03-08T08:26:37.169 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1fb5d10608dccee73edd6ef8a2c7b804a097c324 2024-03-08T08:26:37.312 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T08:26:37.339 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-1670.g1fb5d106.el9 2024-03-08T08:26:37.339 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1670.g1fb5d106.el9 2024-03-08T08:26:37.339 INFO:teuthology.task.install:The correct ceph version 19.0.0-1670.g1fb5d106 is installed. 2024-03-08T08:26:37.341 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-08T08:26:37.341 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:37.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T08:26:37.371 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:26:37.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T08:26:37.414 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-08T08:26:37.415 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:37.415 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T08:26:37.442 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T08:26:37.509 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:26:37.509 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T08:26:37.540 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T08:26:37.608 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-08T08:26:37.609 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:37.609 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T08:26:37.635 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T08:26:37.701 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:26:37.702 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T08:26:37.731 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T08:26:37.799 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-08T08:26:37.799 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:37.799 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T08:26:37.825 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T08:26:37.892 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:26:37.892 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T08:26:37.919 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T08:26:37.986 INFO:teuthology.run_tasks:Running task ceph... 2024-03-08T08:26:38.067 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-08T08:26:38.068 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2024-03-08T08:26:38.070 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2024-03-08T08:26:38.097 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-08T08:26:38.097 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-08T08:26:38.137 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-08T08:26:38.170 INFO:tasks.ceph:Creating extra log directories... 2024-03-08T08:26:38.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-08T08:26:38.203 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-08T08:26:38.245 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-08T08:26:38.245 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-08T08:26:38.245 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-08T08:26:38.246 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:38.246 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-08T08:26:38.284 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-08T08:26:38.401 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-08T08:26:38.428 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:26:38.428 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-08T08:26:38.443 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-08T08:26:38.561 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-08T08:26:38.593 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-08T08:26:38.593 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-08T08:26:38.593 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506', 'branch': 'main', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}', 'email': 'jcollin@redhat.com', 'first_in_suite': False, 'job_id': '7585506', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-jcollin-test_per-client-perf-counters', 'sha1': 'f6f80007dcf9e039bb253dd50a92ba3c71c6943a'}}, 'owner': 'scheduled_jcollin@teuthology', 'priority': 10, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7204, 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-jcollin-test_per-client-perf-counters', 'suite_path': '/home/teuthworker/src/github.com_joscollin_ceph_f6f80007dcf9e039bb253dd50a92ba3c71c6943a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/joscollin/ceph.git', 'suite_sha1': 'f6f80007dcf9e039bb253dd50a92ba3c71c6943a', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi105.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKTYHwn2ct8CPgIFzcYR/P//Mbb19XWmlaFVPA0yaGWofg+DIDzwNQl46giMKuzPAIQPFxspymm1LHkkDBYnbCw=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqi72noN/IPdTZxLSwikM+xHAuFgsGYAZR4z2w1+1k2ffghjOr7RGkIowApJ83KB6CWEXWcIZfnN+TuORTPdEY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'missing required features'], 'sha1': '1fb5d10608dccee73edd6ef8a2c7b804a097c324', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_admin.TestLabeledPerfCounters']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-08_08:09:40', 'tube': 'smithi', 'user': 'jcollin', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-08T08:26:38.594 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-08T08:26:38.596 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-08T08:26:38.644 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2024-03-08T08:26:38.651 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2024-03-08T08:26:38.710 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:38.711 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T08:26:38.730 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T08:26:38.730 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-03-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-08 08:26:28.810254918 +0000 2024-03-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-08 08:23:40.271547958 +0000 2024-03-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-08 08:23:40.271547958 +0000 2024-03-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-08 08:23:40.271547958 +0000 2024-03-08T08:26:38.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T08:26:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-08T08:26:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-08T08:26:38.851 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000202553 s, 2.5 MB/s 2024-03-08T08:26:38.852 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T08:26:38.908 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-03-08T08:26:38.963 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T08:26:38.963 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T08:26:38.963 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-08T08:26:38.963 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T08:26:38.963 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T08:26:38.963 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-08 08:26:28.810254918 +0000 2024-03-08T08:26:38.964 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-08 08:23:40.621541215 +0000 2024-03-08T08:26:38.964 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-08 08:23:40.621541215 +0000 2024-03-08T08:26:38.964 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-08 08:23:40.621541215 +0000 2024-03-08T08:26:38.964 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T08:26:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-08T08:26:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-08T08:26:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000203976 s, 2.5 MB/s 2024-03-08T08:26:39.030 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T08:26:39.086 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-08 08:26:28.810254918 +0000 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-08 08:23:40.966534568 +0000 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-08 08:23:40.966534568 +0000 2024-03-08T08:26:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-08 08:23:40.966534568 +0000 2024-03-08T08:26:39.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T08:26:39.205 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-08T08:26:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-08T08:26:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000206413 s, 2.5 MB/s 2024-03-08T08:26:39.207 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T08:26:39.263 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-03-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-08 08:26:28.811254898 +0000 2024-03-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-08 08:23:41.492524434 +0000 2024-03-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-08 08:23:41.492524434 +0000 2024-03-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-08 08:23:41.492524434 +0000 2024-03-08T08:26:39.319 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T08:26:39.384 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-08T08:26:39.384 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-08T08:26:39.384 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000198094 s, 2.6 MB/s 2024-03-08T08:26:39.385 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T08:26:39.441 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:39.441 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:26:39.441 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T08:26:39.457 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T08:26:39.457 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:25:56.870064640 +0000 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:23:40.276548435 +0000 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:23:40.276548435 +0000 2024-03-08T08:26:39.512 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:23:40.276548435 +0000 2024-03-08T08:26:39.512 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T08:26:39.577 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-08T08:26:39.577 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-08T08:26:39.577 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000221506 s, 2.3 MB/s 2024-03-08T08:26:39.579 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T08:26:39.634 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-03-08T08:26:39.689 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T08:26:39.689 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T08:26:39.689 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-08T08:26:39.690 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T08:26:39.690 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T08:26:39.690 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:25:56.870064640 +0000 2024-03-08T08:26:39.690 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:23:40.614542761 +0000 2024-03-08T08:26:39.690 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:23:40.614542761 +0000 2024-03-08T08:26:39.690 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:23:40.614542761 +0000 2024-03-08T08:26:39.690 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T08:26:39.755 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-08T08:26:39.755 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-08T08:26:39.755 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000177526 s, 2.9 MB/s 2024-03-08T08:26:39.756 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T08:26:39.811 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:25:56.870064640 +0000 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:23:40.966536852 +0000 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:23:40.966536852 +0000 2024-03-08T08:26:39.866 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:23:40.966536852 +0000 2024-03-08T08:26:39.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T08:26:39.931 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-08T08:26:39.931 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-08T08:26:39.931 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000210183 s, 2.4 MB/s 2024-03-08T08:26:39.932 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T08:26:39.988 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-03-08T08:26:40.042 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T08:26:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T08:26:40.043 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-08T08:26:40.043 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T08:26:40.043 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T08:26:40.043 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:25:56.871064624 +0000 2024-03-08T08:26:40.043 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:23:41.310531078 +0000 2024-03-08T08:26:40.043 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:23:41.310531078 +0000 2024-03-08T08:26:40.043 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:23:41.310531078 +0000 2024-03-08T08:26:40.043 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T08:26:40.108 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-08T08:26:40.108 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-08T08:26:40.108 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000174467 s, 2.9 MB/s 2024-03-08T08:26:40.109 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T08:26:40.164 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:40.164 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-08T08:26:40.164 INFO:tasks.ceph:Generating config... 2024-03-08T08:26:40.165 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-08T08:26:40.166 INFO:tasks.ceph:[client] debug client = 20 2024-03-08T08:26:40.166 INFO:tasks.ceph:[client] debug ms = 1 2024-03-08T08:26:40.166 INFO:tasks.ceph:[client] pid file = 2024-03-08T08:26:40.166 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-08T08:26:40.166 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-08T08:26:40.166 INFO:tasks.ceph:[global] lockdep = True 2024-03-08T08:26:40.166 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-08T08:26:40.166 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-08T08:26:40.166 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-08T08:26:40.166 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-08T08:26:40.166 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-08T08:26:40.166 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-08T08:26:40.166 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-08T08:26:40.166 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-08T08:26:40.166 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-08T08:26:40.167 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-08T08:26:40.167 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-08T08:26:40.167 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-08T08:26:40.167 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-08T08:26:40.167 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-08T08:26:40.167 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-08T08:26:40.167 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-08T08:26:40.167 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-08T08:26:40.167 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-08T08:26:40.167 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-08T08:26:40.167 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-08T08:26:40.167 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-08T08:26:40.167 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-08T08:26:40.167 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-08T08:26:40.167 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-08T08:26:40.167 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-08T08:26:40.168 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-08T08:26:40.168 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-08T08:26:40.168 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-08T08:26:40.168 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-08T08:26:40.168 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-08T08:26:40.168 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-08T08:26:40.168 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-08T08:26:40.168 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-08T08:26:40.168 INFO:tasks.ceph:Setting up mon.a... 2024-03-08T08:26:40.168 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.keyring 2024-03-08T08:26:40.228 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-08T08:26:40.274 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T08:26:40.300 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.105'), ('mon.b', '172.21.15.144'), ('mon.c', '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]')] 2024-03-08T08:26:40.300 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.105,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]', 'lockdep': True, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-08T08:26:40.301 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:40.301 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-08T08:26:40.356 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.105 --add b 172.21.15.144 --addv c '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:26:40.439 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: generated fsid 1afff3d7-46c3-4cd2-9212-c89ee069e495 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = quincy 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 0 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:fsid 1afff3d7-46c3-4cd2-9212-c89ee069e495 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:last_changed 2024-03-08T08:26:40.439488+0000 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-03-08T08:26:40.439488+0000 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:min_mon_release 17 (quincy) 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:election_strategy: 1 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:2: [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] mon.c 2024-03-08T08:26:40.440 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-08T08:26:40.441 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-08T08:26:40.496 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1afff3d7-46c3-4cd2-9212-c89ee069e495... 2024-03-08T08:26:40.497 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-08T08:26:40.539 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-08T08:26:40.586 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-03-08T08:26:40.586 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2024-03-08T08:26:40.586 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-08T08:26:40.586 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2024-03-08T08:26:40.586 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.586 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2024-03-08T08:26:40.587 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.588 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.105,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: lockdep = True 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on skipped message = False 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 1afff3d7-46c3-4cd2-9212-c89ee069e495 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2024-03-08T08:26:40.589 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug trim objects = true 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2024-03-08T08:26:40.590 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2024-03-08T08:26:40.591 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2024-03-08T08:26:40.592 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-03-08T08:26:40.593 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2024-03-08T08:26:40.594 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-03-08T08:26:40.595 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2024-03-08T08:26:40.596 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.597 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.598 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.599 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.105,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: lockdep = True 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on skipped message = False 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 1afff3d7-46c3-4cd2-9212-c89ee069e495 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-03-08T08:26:40.600 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-03-08T08:26:40.601 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-03-08T08:26:40.602 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-03-08T08:26:40.603 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-03-08T08:26:40.604 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-03-08T08:26:40.605 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-03-08T08:26:40.606 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-03-08T08:26:40.607 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-03-08T08:26:40.609 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-08T08:26:40.609 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-08T08:26:40.703 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-08T08:26:40.704 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:40.704 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-08T08:26:40.718 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:40.718 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-08T08:26:40.773 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:26:40.773 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:40.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-08T08:26:40.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T08:26:40.851 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:26:40.851 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:26:40.906 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:26:40.906 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:26:40.906 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-08T08:26:40.906 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-08T08:26:40.945 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:26:40.945 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:26:41.001 INFO:tasks.ceph:Setting up mon nodes... 2024-03-08T08:26:41.001 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-08T08:26:41.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-08T08:26:41.055 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-08T08:26:41.057 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-08T08:26:41.117 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-08T08:26:41.119 INFO:tasks.ceph:Setting up mds nodes... 2024-03-08T08:26:41.120 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-08T08:26:41.174 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-08T08:26:41.176 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-08T08:26:41.203 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-08T08:26:41.296 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-08T08:26:41.299 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-08T08:26:41.325 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-08T08:26:41.379 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-08T08:26:41.382 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-08T08:26:41.408 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-08T08:26:41.503 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-08T08:26:41.506 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-08T08:26:41.533 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-08T08:26:41.533 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-08T08:26:41.575 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-08T08:26:41.590 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-08T08:26:41.672 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-08T08:26:41.687 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-08T08:26:41.729 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-08T08:26:41.744 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-08T08:26:41.826 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-08T08:26:41.841 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-08T08:26:41.842 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-08T08:26:41.842 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-08T08:26:41.868 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:41.868 INFO:tasks.ceph:role: osd.0 2024-03-08T08:26:41.868 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:26:41.868 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-08T08:26:41.942 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T08:26:41.942 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T08:26:41.942 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T08:26:41.942 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T08:26:41.942 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T08:26:41.943 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-03-08T08:26:41.943 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T08:26:41.943 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T08:26:41.943 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T08:26:41.943 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T08:26:42.462 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-03-08T08:26:42.464 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-03-08T08:26:42.464 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-08T08:26:42.603 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-08T08:26:42.636 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-08T08:26:42.703 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:42.703 INFO:tasks.ceph:role: osd.1 2024-03-08T08:26:42.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:26:42.703 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-08T08:26:42.777 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T08:26:42.778 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T08:26:43.222 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-03-08T08:26:43.224 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-03-08T08:26:43.224 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-08T08:26:43.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-08T08:26:43.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-08T08:26:43.413 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:43.413 INFO:tasks.ceph:role: osd.2 2024-03-08T08:26:43.413 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:26:43.413 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-08T08:26:43.487 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T08:26:43.487 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T08:26:43.487 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T08:26:43.487 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T08:26:43.488 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T08:26:43.488 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-03-08T08:26:43.488 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T08:26:43.488 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T08:26:43.488 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T08:26:43.488 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T08:26:43.975 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-03-08T08:26:43.977 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-03-08T08:26:43.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-08T08:26:44.116 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-08T08:26:44.148 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-08T08:26:44.216 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:44.216 INFO:tasks.ceph:role: osd.3 2024-03-08T08:26:44.216 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:26:44.216 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-08T08:26:44.292 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T08:26:44.292 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T08:26:44.292 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T08:26:44.292 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T08:26:44.292 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T08:26:44.292 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-03-08T08:26:44.292 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T08:26:44.292 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T08:26:44.293 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T08:26:44.293 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T08:26:44.803 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-03-08T08:26:44.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-03-08T08:26:44.805 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-08T08:26:44.938 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-08T08:26:44.970 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:26:45.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:45.067+0000 7f8df7a8b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-08T08:26:45.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:45.067+0000 7f8df7a8b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-08T08:26:45.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:45.067+0000 7f8df7a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T08:26:45.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:45.067+0000 7f8df7a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T08:26:45.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:45.067+0000 7f8df7a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-08T08:26:45.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:45.068+0000 7f8df7a8b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-08T08:26:47.921 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-08T08:26:47.953 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:26:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:48.042+0000 7fb42480f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-08T08:26:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:48.043+0000 7fb42480f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-08T08:26:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:48.043+0000 7fb42480f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T08:26:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:48.043+0000 7fb42480f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T08:26:48.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:48.043+0000 7fb42480f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-08T08:26:48.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:48.043+0000 7fb42480f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-08T08:26:50.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-08T08:26:50.925 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:26:51.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:51.011+0000 7f8e91a24640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-08T08:26:51.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:51.012+0000 7f8e91a24640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-08T08:26:51.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:51.012+0000 7f8e91a24640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T08:26:51.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:51.012+0000 7f8e91a24640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T08:26:51.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:51.012+0000 7f8e91a24640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-08T08:26:51.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:51.012+0000 7f8e91a24640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-08T08:26:53.869 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-08T08:26:53.900 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:53.990+0000 7fbc7a068640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:53.991+0000 7fbc7a068640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:53.991+0000 7fbc7a068640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:53.991+0000 7fbc7a068640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:53.991+0000 7fbc7a068640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:26:53.991+0000 7fbc7a068640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-08T08:26:56.841 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-08T08:26:56.872 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-08T08:26:56.904 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:56.904 INFO:tasks.ceph:role: osd.4 2024-03-08T08:26:56.904 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:26:56.904 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-08T08:26:56.981 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T08:26:56.981 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T08:26:56.981 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T08:26:56.981 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T08:26:56.981 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T08:26:56.981 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-03-08T08:26:56.981 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T08:26:56.981 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T08:26:56.982 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T08:26:56.982 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-03-08T08:26:57.504 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-03-08T08:26:57.504 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-08T08:26:57.653 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-08T08:26:57.689 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-08T08:26:57.758 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:57.759 INFO:tasks.ceph:role: osd.5 2024-03-08T08:26:57.759 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:26:57.759 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T08:26:57.834 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T08:26:58.283 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-03-08T08:26:58.285 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-03-08T08:26:58.286 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-08T08:26:58.368 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-08T08:26:58.440 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-08T08:26:58.506 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:58.521 INFO:tasks.ceph:role: osd.6 2024-03-08T08:26:58.521 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:26:58.521 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-08T08:26:58.580 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-03-08T08:26:59.081 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-03-08T08:26:59.081 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-08T08:26:59.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-08T08:26:59.235 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-08T08:26:59.303 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-08T08:26:59.303 INFO:tasks.ceph:role: osd.7 2024-03-08T08:26:59.303 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:26:59.303 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-08T08:26:59.377 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-08T08:26:59.377 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-08T08:26:59.377 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-08T08:26:59.377 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-08T08:26:59.377 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-08T08:26:59.378 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-03-08T08:26:59.378 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-08T08:26:59.378 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-08T08:26:59.378 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-08T08:26:59.378 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-08T08:26:59.875 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-03-08T08:26:59.877 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-03-08T08:26:59.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-08T08:27:00.007 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-08T08:27:00.041 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:27:00.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:00.131+0000 7fd20ad8a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-08T08:27:00.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:00.132+0000 7fd20ad8a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-08T08:27:00.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:00.132+0000 7fd20ad8a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T08:27:00.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:00.132+0000 7fd20ad8a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T08:27:00.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:00.132+0000 7fd20ad8a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-08T08:27:00.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:00.134+0000 7fd20ad8a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-08T08:27:02.985 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-08T08:27:03.015 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:03.103+0000 7fecc361d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:03.104+0000 7fecc361d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:03.104+0000 7fecc361d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:03.104+0000 7fecc361d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:03.104+0000 7fecc361d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:03.104+0000 7fecc361d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-08T08:27:05.953 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-08T08:27:05.983 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:27:06.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:06.070+0000 7fa3311bb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-08T08:27:06.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:06.070+0000 7fa3311bb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-08T08:27:06.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:06.070+0000 7fa3311bb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T08:27:06.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:06.070+0000 7fa3311bb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T08:27:06.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:06.070+0000 7fa3311bb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-08T08:27:06.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:06.071+0000 7fa3311bb640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-08T08:27:08.595 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:27:08.597 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:27:08.920 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-08T08:27:08.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:27:09.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:09.034+0000 7fc17ec15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-08T08:27:09.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:09.034+0000 7fc17ec15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-08T08:27:09.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:09.034+0000 7fc17ec15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T08:27:09.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:09.034+0000 7fc17ec15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T08:27:09.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:09.034+0000 7fc17ec15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-08T08:27:09.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:09.035+0000 7fc17ec15640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-08T08:27:11.889 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-08T08:27:11.920 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-08T08:27:11.920 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:11.920 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-08T08:27:11.952 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:11.952 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-08T08:27:12.022 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:12.022 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-08T08:27:12.090 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:12.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-08T08:27:12.158 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:12.159 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-08T08:27:12.226 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:12.226 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-08T08:27:12.292 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:12.293 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-08T08:27:12.359 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.359 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-08T08:27:12.390 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.391 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-08T08:27:12.462 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.462 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-08T08:27:12.533 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.533 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-08T08:27:12.602 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.602 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-08T08:27:12.670 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.670 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-08T08:27:12.738 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.738 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-08T08:27:12.805 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:12.806 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-08T08:27:12.821 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:12.821 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-08T08:27:12.877 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.877 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-08T08:27:12.893 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:12.893 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-08T08:27:12.951 INFO:tasks.ceph:Adding keys to all mons... 2024-03-08T08:27:12.951 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-08T08:27:12.953 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-08T08:27:12.994 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2024-03-08T08:27:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+pleaZMAxAAaBcd2a/W/2YV1SCHxNNT4Q== 2024-03-08T08:27:12.994 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-03-08T08:27:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+plo/lhChAAG3AxM32YObwo6H9eS31x1g== 2024-03-08T08:27:12.994 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-03-08T08:27:12.994 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+pla7qwERAAeVrSqF6zglXuWKRqC1e9hg== 2024-03-08T08:27:12.994 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDFy+pl8FscBBAA8TFf1gnifLU4Bdu/8Zp2AQ== 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDIy+plYj2nAhAAA+swRGEJkSYG/A2Bjeyztg== 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDLy+plLNvNABAA+ErHHuno91i45lPDcQVlbA== 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDNy+plG8YpOxAAolVB6FJlT8X/cuKRa2E9+g== 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+plV4P+BhAAs7ZptbBbuQM8B7SnOAGOnQ== 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+pl1r+gFhAA7lND37bRmxyruezGya8zZg== 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+pls20EHhAAmSsOB414olq6Vz1VcC/uvQ== 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2024-03-08T08:27:12.995 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDUy+plKvXrBxAAWA+vmWj1l68pyQ81i6Gxhg== 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDXy+pl+R1NBhAARFNkrHGsiUCbQyobuuoWTg== 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDay+plTd5DBBAAMgKGvfqMAw8OyFpZcB5ryg== 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDdy+plum0eAhAAV2lbgyOIrv8Z6m3tdxNdSA== 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+pl4CRKIhAAtr/P93sD/1absmF733EcnQ== 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout:[client.2] 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+pl2bATKBAA1L5Hmjhd3LmVbLiOXqyCcg== 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+plwux9KxAArAvxh6JfHfMYd8kHKMOfXw== 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout:[client.3] 2024-03-08T08:27:12.996 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+pl/WFHMRAA5pEX7jDEu6LBea+B7ULUiQ== 2024-03-08T08:27:13.016 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.y] 2024-03-08T08:27:13.016 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+pleaZMAxAAaBcd2a/W/2YV1SCHxNNT4Q== 2024-03-08T08:27:13.016 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-03-08T08:27:13.016 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+plo/lhChAAG3AxM32YObwo6H9eS31x1g== 2024-03-08T08:27:13.016 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+pla7qwERAAeVrSqF6zglXuWKRqC1e9hg== 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDFy+pl8FscBBAA8TFf1gnifLU4Bdu/8Zp2AQ== 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDIy+plYj2nAhAAA+swRGEJkSYG/A2Bjeyztg== 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDLy+plLNvNABAA+ErHHuno91i45lPDcQVlbA== 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDNy+plG8YpOxAAolVB6FJlT8X/cuKRa2E9+g== 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+plV4P+BhAAs7ZptbBbuQM8B7SnOAGOnQ== 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+pl1r+gFhAA7lND37bRmxyruezGya8zZg== 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-03-08T08:27:13.017 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+pls20EHhAAmSsOB414olq6Vz1VcC/uvQ== 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDUy+plKvXrBxAAWA+vmWj1l68pyQ81i6Gxhg== 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDXy+pl+R1NBhAARFNkrHGsiUCbQyobuuoWTg== 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.6] 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDay+plTd5DBBAAMgKGvfqMAw8OyFpZcB5ryg== 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.7] 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDdy+plum0eAhAAV2lbgyOIrv8Z6m3tdxNdSA== 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+pl4CRKIhAAtr/P93sD/1absmF733EcnQ== 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout:[client.2] 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+pl2bATKBAA1L5Hmjhd3LmVbLiOXqyCcg== 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+plwux9KxAArAvxh6JfHfMYd8kHKMOfXw== 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout:[client.3] 2024-03-08T08:27:13.018 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBy+pl/WFHMRAA5pEX7jDEu6LBea+B7ULUiQ== 2024-03-08T08:27:13.019 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T08:27:13.040 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T08:27:13.113 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T08:27:13.116 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T08:27:13.170 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T08:27:13.172 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T08:27:13.226 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.228 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.280 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.334 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.336 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.388 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.390 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.441 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T08:27:13.443 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-08T08:27:13.495 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T08:27:13.497 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T08:27:13.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T08:27:13.555 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-08T08:27:13.609 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.611 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.663 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.716 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.719 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.770 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-08T08:27:13.824 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T08:27:13.826 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T08:27:13.880 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T08:27:13.883 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T08:27:13.937 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T08:27:13.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T08:27:13.994 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T08:27:13.995 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-08T08:27:14.050 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-08T08:27:14.050 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-08T08:27:14.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T08:27:14.172 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-08T08:27:14.198 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-08T08:27:14.226 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T08:27:14.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-08T08:27:14.348 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-08T08:27:14.415 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-08T08:27:14.511 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-08T08:27:14.537 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:27:14.539 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-08T08:27:14.592 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-08T08:27:14.592 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-08T08:27:14.592 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-08T08:27:14.594 INFO:tasks.ceph.mon.a:Started 2024-03-08T08:27:14.595 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-08T08:27:14.595 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-08T08:27:14.634 INFO:tasks.ceph.mon.b:Started 2024-03-08T08:27:14.634 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-08T08:27:14.634 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-08T08:27:14.636 INFO:tasks.ceph.mon.c:Started 2024-03-08T08:27:14.636 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-08T08:27:14.636 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-08T08:27:14.637 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-08T08:27:14.639 INFO:tasks.ceph.mgr.y:Started 2024-03-08T08:27:14.639 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-08T08:27:14.639 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-08T08:27:14.641 INFO:tasks.ceph.mgr.x:Started 2024-03-08T08:27:14.641 INFO:tasks.ceph:Setting crush tunables to default 2024-03-08T08:27:14.641 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd crush tunables default 2024-03-08T08:27:14.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.740+0000 7f37dba88640 1 Processor -- start 2024-03-08T08:27:14.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.740+0000 7f37dba88640 1 -- start start 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37dba88640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f37d4161470 0x7f37d406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37dba88640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 0x7f37d40d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37dba88640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d40d48a0 con 0x7f37d4161850 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37dba88640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d40d2ba0 con 0x7f37d406bcb0 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37dba88640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d40d4b80 con 0x7f37d4161470 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37da025640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 0x7f37d40d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37da025640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 0x7f37d40d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35876/0 (socket says 172.21.15.105:35876) 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37da025640 1 -- 172.21.15.105:0/169467884 learned_addr learned my addr 172.21.15.105:0/169467884 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37da025640 1 -- 172.21.15.105:0/169467884 >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 msgr2=0x7f37d40d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37da025640 1 -- 172.21.15.105:0/169467884 >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 msgr2=0x7f37d40d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37da025640 1 --2- 172.21.15.105:0/169467884 >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 0x7f37d40d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37da025640 1 --2- 172.21.15.105:0/169467884 >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 0x7f37d40d2660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37d9824640 1 --2- 172.21.15.105:0/169467884 >> v2:172.21.15.144:3300/0 conn(0x7f37d4161470 0x7f37d406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37d9824640 1 -- 172.21.15.105:0/169467884 >> v2:172.21.15.144:3300/0 conn(0x7f37d4161470 msgr2=0x7f37d406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-08T08:27:14.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37d9824640 1 -- 172.21.15.105:0/169467884 >> v2:172.21.15.144:3300/0 conn(0x7f37d4161470 msgr2=0x7f37d406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-08T08:27:14.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37d9824640 1 --2- 172.21.15.105:0/169467884 >> v2:172.21.15.144:3300/0 conn(0x7f37d4161470 0x7f37d406c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T08:27:14.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.741+0000 7f37d9824640 1 --2- 172.21.15.105:0/169467884 >> v2:172.21.15.144:3300/0 conn(0x7f37d4161470 0x7f37d406c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T08:27:14.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.936+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252559130 0 0) 0x7f37d40d48a0 con 0x7f37d4161850 2024-03-08T08:27:14.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.936+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c0003520 con 0x7f37d4161850 2024-03-08T08:27:14.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.936+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138889686 0 0) 0x7f37c0003520 con 0x7f37d4161850 2024-03-08T08:27:14.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 >> v2:172.21.15.144:3300/0 conn(0x7f37d4161470 msgr2=0x7f37d406c830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T08:27:14.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 --2- 172.21.15.105:0/169467884 >> v2:172.21.15.144:3300/0 conn(0x7f37d4161470 0x7f37d406c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:14.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 msgr2=0x7f37d40d2660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T08:27:14.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 --2- 172.21.15.105:0/169467884 >> v2:172.21.15.105:3300/0 conn(0x7f37d406bcb0 0x7f37d40d2660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:14.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d406c170 con 0x7f37d4161850 2024-03-08T08:27:14.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f37c4001670 con 0x7f37d4161850 2024-03-08T08:27:14.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 >> v1:172.21.15.105:6789/0 conn(0x7f37d4161850 legacy=0x7f37d406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0003a00 0x7f37c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0004800 0x7f37c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c0005210 0x7f37c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40d2ba0 con 0x7f37c0005210 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40d4b80 con 0x7f37c0003a00 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c000c1c0 con 0x7f37c0004800 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37da025640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c0005210 0x7f37c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d9824640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0003a00 0x7f37c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d9023640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0004800 0x7f37c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37da025640 1 -- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0004800 msgr2=0x7f37c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37da025640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0004800 0x7f37c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37da025640 1 -- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0003a00 msgr2=0x7f37c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37da025640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c0003a00 0x7f37c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37da025640 1 -- 172.21.15.105:0/169467884 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c0003520 con 0x7f37c0005210 2024-03-08T08:27:14.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.937+0000 7f37d9023640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0004800 0x7f37c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:14.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.938+0000 7f37da025640 1 -- 172.21.15.105:0/169467884 reap_dead start 2024-03-08T08:27:14.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.938+0000 7f37da025640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c0005210 0x7f37c000b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009890 tx=0x7f37c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affe935a25016e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:14.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.938+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0034070 con 0x7f37c0005210 2024-03-08T08:27:14.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.938+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d00306c0 con 0x7f37c0005210 2024-03-08T08:27:14.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.938+0000 7f37d8822640 1 -- 172.21.15.105:0/169467884 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d000f040 con 0x7f37c0005210 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.939+0000 7f37dba88640 1 -- 172.21.15.105:0/169467884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c0005210 msgr2=0x7f37c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.939+0000 7f37dba88640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c0005210 0x7f37c000b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009890 tx=0x7f37c0005830 comp rx=0 tx=0).stop 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.939+0000 7f37dba88640 1 -- 172.21.15.105:0/169467884 shutdown_connections 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.939+0000 7f37dba88640 1 --2- 172.21.15.105:0/169467884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c0005210 0x7f37c000b9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.939+0000 7f37dba88640 1 -- 172.21.15.105:0/169467884 >> 172.21.15.105:0/169467884 conn(0x7f37d40d3ce0 msgr2=0x7f37d40d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.939+0000 7f37dba88640 1 -- 172.21.15.105:0/169467884 shutdown_connections 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.939+0000 7f37dba88640 1 -- 172.21.15.105:0/169467884 wait complete. 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.941+0000 7f37dba88640 1 Processor -- start 2024-03-08T08:27:14.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.941+0000 7f37dba88640 1 -- start start 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37dba88640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0005210 0x7f37d40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37dba88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d415d660 0x7f37d4159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37dba88640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d415a5b0 0x7f37d40b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d415d660 0x7f37d4159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37dba88640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d406cf40 con 0x7f37d415d660 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37dba88640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d406d370 con 0x7f37d415a5b0 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37dba88640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40cc030 con 0x7f37c0005210 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d415d660 0x7f37d4159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35928/0 (socket says 172.21.15.105:35928) 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 -- 172.21.15.105:0/3393711093 learned_addr learned my addr 172.21.15.105:0/3393711093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9824640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0005210 0x7f37d40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:14.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37da025640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d415a5b0 0x7f37d40b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 -- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0005210 msgr2=0x7f37d40c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0005210 0x7f37d40c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 -- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d415a5b0 msgr2=0x7f37d40b8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d415a5b0 0x7f37d40b8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 -- 172.21.15.105:0/3393711093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d415af20 con 0x7f37d415d660 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37da025640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d415a5b0 0x7f37d40b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.942+0000 7f37d9023640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d415d660 0x7f37d4159d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37c400e780 tx=0x7f37c400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.943+0000 7f37ca7fc640 1 -- 172.21.15.105:0/3393711093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c40050d0 con 0x7f37d415d660 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.943+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d415b200 con 0x7f37d415d660 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.943+0000 7f37ca7fc640 1 -- 172.21.15.105:0/3393711093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c4018070 con 0x7f37d415d660 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.943+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d40cc260 con 0x7f37d415d660 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.943+0000 7f37ca7fc640 1 -- 172.21.15.105:0/3393711093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4005470 con 0x7f37d415d660 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.943+0000 7f37ca7fc640 1 -- 172.21.15.105:0/3393711093 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f37c400c6c0 con 0x7f37d415d660 2024-03-08T08:27:14.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.943+0000 7f37ca7fc640 1 -- 172.21.15.105:0/3393711093 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f37c4003d20 con 0x7f37d415d660 2024-03-08T08:27:14.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.944+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d4075120 con 0x7f37d415d660 2024-03-08T08:27:14.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:14.948+0000 7f37ca7fc640 1 -- 172.21.15.105:0/3393711093 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f37c400cb90 con 0x7f37d415d660 2024-03-08T08:27:15.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.014+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f37d40b9b10 con 0x7f37d415d660 2024-03-08T08:27:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.016+0000 7f37ca7fc640 1 -- 172.21.15.105:0/3393711093 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f37c400cd70 con 0x7f37d415d660 2024-03-08T08:27:15.017 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d415d660 msgr2=0x7f37d4159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d415d660 0x7f37d4159d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37c400e780 tx=0x7f37c400ec40 comp rx=0 tx=0).stop 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 shutdown_connections 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d415a5b0 0x7f37d40b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d415d660 0x7f37d4159d00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 --2- 172.21.15.105:0/3393711093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0005210 0x7f37d40c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 >> 172.21.15.105:0/3393711093 conn(0x7f37d40d3ce0 msgr2=0x7f37d40b74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 shutdown_connections 2024-03-08T08:27:15.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:15.017+0000 7f37dba88640 1 -- 172.21.15.105:0/3393711093 wait complete. 2024-03-08T08:27:15.029 INFO:tasks.ceph:check_enable_crimson: False 2024-03-08T08:27:15.029 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-08T08:27:15.029 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:15.029 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-08T08:27:15.055 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:15.055 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-08T08:27:15.122 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:15.122 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-08T08:27:15.188 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:15.188 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-08T08:27:15.256 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:15.256 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-08T08:27:15.283 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:15.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-08T08:27:15.352 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:15.352 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-08T08:27:15.410 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:15.409+0000 7ff3f426c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T08:27:15.419 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:27:15.419 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-08T08:27:15.423 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:15.422+0000 7fdacde13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T08:27:15.449 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 32540e82-9419-45ae-840e-488ebfd5fdeb 0 2024-03-08T08:27:15.463 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:15.463+0000 7ff3f426c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T08:27:15.480 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:15.478+0000 7fdacde13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T08:27:15.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.577+0000 7fa14bab5640 1 Processor -- start 2024-03-08T08:27:15.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.578+0000 7fa14bab5640 1 -- start start 2024-03-08T08:27:15.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.578+0000 7fa14bab5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406b390 0x7fa14406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.578+0000 7fa14bab5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14406f830 con 0x7fa14415df20 2024-03-08T08:27:15.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.578+0000 7fa14bab5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14406fad0 con 0x7fa1440b5990 2024-03-08T08:27:15.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.578+0000 7fa14bab5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14406dbb0 con 0x7fa14406b390 2024-03-08T08:27:15.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.579+0000 7fa14a052640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa1440b5990 0x7fa1440ad970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:45578/0 (socket says 172.21.15.144:45578) 2024-03-08T08:27:15.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.579+0000 7fa14a052640 1 -- 172.21.15.144:0/514740152 learned_addr learned my addr 172.21.15.144:0/514740152 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-03-08T08:27:15.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.579+0000 7fa149050640 1 --2- 172.21.15.144:0/514740152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406b390 0x7fa14406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.579+0000 7fa14884f640 1 -- 172.21.15.144:0/514740152 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846525413 0 0) 0x7fa14406fad0 con 0x7fa1440b5990 2024-03-08T08:27:15.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.579+0000 7fa14884f640 1 -- 172.21.15.144:0/514740152 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa130003520 con 0x7fa1440b5990 2024-03-08T08:27:15.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.579+0000 7fa149050640 1 -- 172.21.15.144:0/514740152 >> v1:172.21.15.144:6789/0 conn(0x7fa1440b5990 legacy=0x7fa1440ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.579+0000 7fa149050640 1 -- 172.21.15.144:0/514740152 >> v1:172.21.15.105:6789/0 conn(0x7fa14415df20 legacy=0x7fa14406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.579+0000 7fa149050640 1 -- 172.21.15.144:0/514740152 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14406fdb0 con 0x7fa14406b390 2024-03-08T08:27:15.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa149050640 1 --2- 172.21.15.144:0/514740152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406b390 0x7fa14406c8b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa1340097d0 tx=0x7fa13402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72cef3af9686fd3d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:15.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14884f640 1 -- 172.21.15.144:0/514740152 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134035020 con 0x7fa14406b390 2024-03-08T08:27:15.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14884f640 1 -- 172.21.15.144:0/514740152 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa134031800 con 0x7fa14406b390 2024-03-08T08:27:15.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14884f640 1 -- 172.21.15.144:0/514740152 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134031bd0 con 0x7fa14406b390 2024-03-08T08:27:15.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14bab5640 1 -- 172.21.15.144:0/514740152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406b390 msgr2=0x7fa14406c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14bab5640 1 --2- 172.21.15.144:0/514740152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406b390 0x7fa14406c8b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa1340097d0 tx=0x7fa13402b8a0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14bab5640 1 -- 172.21.15.144:0/514740152 shutdown_connections 2024-03-08T08:27:15.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14bab5640 1 --2- 172.21.15.144:0/514740152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14406b390 0x7fa14406c8b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14bab5640 1 -- 172.21.15.144:0/514740152 >> 172.21.15.144:0/514740152 conn(0x7fa14406bd90 msgr2=0x7fa14406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:15.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14bab5640 1 -- 172.21.15.144:0/514740152 shutdown_connections 2024-03-08T08:27:15.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.580+0000 7fa14bab5640 1 -- 172.21.15.144:0/514740152 wait complete. 2024-03-08T08:27:15.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14bab5640 1 Processor -- start 2024-03-08T08:27:15.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14bab5640 1 -- start start 2024-03-08T08:27:15.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14bab5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa14406b390 0x7fa144142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14bab5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1440b5990 0x7fa14415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14bab5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415df20 0x7fa1440aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14bab5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440ae080 con 0x7fa14406b390 2024-03-08T08:27:15.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14bab5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14406e0e0 con 0x7fa1440b5990 2024-03-08T08:27:15.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14bab5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440ae200 con 0x7fa14415df20 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa149050640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1440b5990 0x7fa14415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa14a052640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415df20 0x7fa1440aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa149050640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1440b5990 0x7fa14415a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33594/0 (socket says 172.21.15.144:33594) 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa149050640 1 -- 172.21.15.144:0/1429801430 learned_addr learned my addr 172.21.15.144:0/1429801430 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.582+0000 7fa149851640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa14406b390 0x7fa144142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa149050640 1 -- 172.21.15.144:0/1429801430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415df20 msgr2=0x7fa1440aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa149050640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415df20 0x7fa1440aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa149050640 1 -- 172.21.15.144:0/1429801430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa14406b390 msgr2=0x7fa144142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa149050640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa14406b390 0x7fa144142e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa149050640 1 -- 172.21.15.144:0/1429801430 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14415b440 con 0x7fa1440b5990 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa14a052640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415df20 0x7fa1440aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa149050640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1440b5990 0x7fa14415a9f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa1340098f0 tx=0x7fa1340046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa13a7fc640 1 -- 172.21.15.144:0/1429801430 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134041070 con 0x7fa1440b5990 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa13a7fc640 1 -- 172.21.15.144:0/1429801430 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13403ad00 con 0x7fa1440b5990 2024-03-08T08:27:15.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14415b720 con 0x7fa1440b5990 2024-03-08T08:27:15.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa13a7fc640 1 -- 172.21.15.144:0/1429801430 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134035040 con 0x7fa1440b5990 2024-03-08T08:27:15.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14415bce0 con 0x7fa1440b5990 2024-03-08T08:27:15.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa13a7fc640 1 -- 172.21.15.144:0/1429801430 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa13404c060 con 0x7fa1440b5990 2024-03-08T08:27:15.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.583+0000 7fa13a7fc640 1 -- 172.21.15.144:0/1429801430 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa13404e4f0 con 0x7fa1440b5990 2024-03-08T08:27:15.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.584+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa144075120 con 0x7fa1440b5990 2024-03-08T08:27:15.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.586+0000 7fa13a7fc640 1 -- 172.21.15.144:0/1429801430 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa13403bb40 con 0x7fa1440b5990 2024-03-08T08:27:15.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.657+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "32540e82-9419-45ae-840e-488ebfd5fdeb", "id": 0} v 0) v1 -- 0x7fa14406ef60 con 0x7fa1440b5990 2024-03-08T08:27:15.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.660+0000 7fa13a7fc640 1 -- 172.21.15.144:0/1429801430 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "32540e82-9419-45ae-840e-488ebfd5fdeb", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa13403bd20 con 0x7fa1440b5990 2024-03-08T08:27:15.661 INFO:teuthology.orchestra.run.smithi144.stdout:0 2024-03-08T08:27:15.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1440b5990 msgr2=0x7fa14415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1440b5990 0x7fa14415a9f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa1340098f0 tx=0x7fa1340046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 shutdown_connections 2024-03-08T08:27:15.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa14415df20 0x7fa1440aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1440b5990 0x7fa14415a9f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 --2- 172.21.15.144:0/1429801430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa14406b390 0x7fa144142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 >> 172.21.15.144:0/1429801430 conn(0x7fa14406bd90 msgr2=0x7fa14415c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:15.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 shutdown_connections 2024-03-08T08:27:15.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.661+0000 7fa14bab5640 1 -- 172.21.15.144:0/1429801430 wait complete. 2024-03-08T08:27:15.672 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 2e4be82f-332a-4c95-8a81-b7a186b0c433 1 2024-03-08T08:27:15.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.772+0000 7ff155172640 1 Processor -- start 2024-03-08T08:27:15.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.772+0000 7ff155172640 1 -- start start 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.772+0000 7ff155172640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff15015bea0 0x7ff1500b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff155172640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff15015df30 0x7ff1500b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff155172640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15015f7b0 con 0x7ff1500b5a60 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff155172640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1500b5ed0 con 0x7ff15015df30 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff155172640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15015e710 con 0x7ff15015bea0 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14ed9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff15015bea0 0x7ff1500b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14ed9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff15015bea0 0x7ff1500b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33602/0 (socket says 172.21.15.144:33602) 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14ed9d640 1 -- 172.21.15.144:0/2840488375 learned_addr learned my addr 172.21.15.144:0/2840488375 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14e59c640 1 --2- 172.21.15.144:0/2840488375 >> v2:172.21.15.105:3300/0 conn(0x7ff15015df30 0x7ff1500b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14ed9d640 1 -- 172.21.15.144:0/2840488375 >> v2:172.21.15.105:3300/0 conn(0x7ff15015df30 msgr2=0x7ff1500b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14ed9d640 1 --2- 172.21.15.144:0/2840488375 >> v2:172.21.15.105:3300/0 conn(0x7ff15015df30 0x7ff1500b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14ed9d640 1 -- 172.21.15.144:0/2840488375 >> v1:172.21.15.105:6789/0 conn(0x7ff1500b5a60 legacy=0x7ff15009baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:15.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14ed9d640 1 -- 172.21.15.144:0/2840488375 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15015e9f0 con 0x7ff15015bea0 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.773+0000 7ff14e59c640 1 --2- 172.21.15.144:0/2840488375 >> v2:172.21.15.105:3300/0 conn(0x7ff15015df30 0x7ff1500b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff14ed9d640 1 --2- 172.21.15.144:0/2840488375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015bea0 0x7ff1500b4e40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff150072010 tx=0x7ff14002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b559dc916b9013 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff14dd9b640 1 -- 172.21.15.144:0/2840488375 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140035020 con 0x7ff15015bea0 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff14dd9b640 1 -- 172.21.15.144:0/2840488375 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff140031870 con 0x7ff15015bea0 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff14dd9b640 1 -- 172.21.15.144:0/2840488375 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140031c40 con 0x7ff15015bea0 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff155172640 1 -- 172.21.15.144:0/2840488375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015bea0 msgr2=0x7ff1500b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff155172640 1 --2- 172.21.15.144:0/2840488375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015bea0 0x7ff1500b4e40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff150072010 tx=0x7ff14002b910 comp rx=0 tx=0).stop 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff155172640 1 -- 172.21.15.144:0/2840488375 shutdown_connections 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff155172640 1 --2- 172.21.15.144:0/2840488375 >> v2:172.21.15.105:3300/0 conn(0x7ff15015df30 0x7ff1500b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff155172640 1 --2- 172.21.15.144:0/2840488375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015bea0 0x7ff1500b4e40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff155172640 1 -- 172.21.15.144:0/2840488375 >> 172.21.15.144:0/2840488375 conn(0x7ff15009aa00 msgr2=0x7ff15009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff155172640 1 -- 172.21.15.144:0/2840488375 shutdown_connections 2024-03-08T08:27:15.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.774+0000 7ff155172640 1 -- 172.21.15.144:0/2840488375 wait complete. 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.776+0000 7ff155172640 1 Processor -- start 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.776+0000 7ff155172640 1 -- start start 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff155172640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1500b5a60 0x7ff150158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff155172640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff15015bea0 0x7ff15006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff155172640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015df30 0x7ff15006d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff155172640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15015f290 con 0x7ff1500b5a60 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff155172640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15015efb0 con 0x7ff15015df30 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff155172640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15015d290 con 0x7ff15015bea0 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff15015bea0 0x7ff15006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015df30 0x7ff15006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff15015bea0 0x7ff15006cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:39132/0 (socket says 172.21.15.144:39132) 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 -- 172.21.15.144:0/1060608055 learned_addr learned my addr 172.21.15.144:0/1060608055 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14ed9d640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1500b5a60 0x7ff150158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 -- 172.21.15.144:0/1060608055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015df30 msgr2=0x7ff15006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015df30 0x7ff15006d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 -- 172.21.15.144:0/1060608055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1500b5a60 msgr2=0x7ff150158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1500b5a60 0x7ff150158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 -- 172.21.15.144:0/1060608055 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150159550 con 0x7ff15015bea0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14f59e640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015df30 0x7ff15006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff14e59c640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff15015bea0 0x7ff15006cac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c7f0 tx=0x7ff14400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff1377fe640 1 -- 172.21.15.144:0/1060608055 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144015070 con 0x7ff15015bea0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff1377fe640 1 -- 172.21.15.144:0/1060608055 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff144007a30 con 0x7ff15015bea0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff150159830 con 0x7ff15015bea0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff1377fe640 1 -- 172.21.15.144:0/1060608055 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144007e00 con 0x7ff15015bea0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.777+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff150159df0 con 0x7ff15015bea0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.778+0000 7ff1377fe640 1 -- 172.21.15.144:0/1060608055 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff144018070 con 0x7ff15015bea0 2024-03-08T08:27:15.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.778+0000 7ff1377fe640 1 -- 172.21.15.144:0/1060608055 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff144011910 con 0x7ff15015bea0 2024-03-08T08:27:15.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.778+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff150075120 con 0x7ff15015bea0 2024-03-08T08:27:15.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.780+0000 7ff1377fe640 1 -- 172.21.15.144:0/1060608055 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff144011bc0 con 0x7ff15015bea0 2024-03-08T08:27:15.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.852+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2e4be82f-332a-4c95-8a81-b7a186b0c433", "id": 1} v 0) v1 -- 0x7ff15015b590 con 0x7ff15015bea0 2024-03-08T08:27:15.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.858+0000 7ff1377fe640 1 -- 172.21.15.144:0/1060608055 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2e4be82f-332a-4c95-8a81-b7a186b0c433", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff14401a450 con 0x7ff15015bea0 2024-03-08T08:27:15.859 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-03-08T08:27:15.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff15015bea0 msgr2=0x7ff15006cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff15015bea0 0x7ff15006cac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c7f0 tx=0x7ff14400cbf0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 shutdown_connections 2024-03-08T08:27:15.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff15015df30 0x7ff15006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff15015bea0 0x7ff15006cac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 --2- 172.21.15.144:0/1060608055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1500b5a60 0x7ff150158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 >> 172.21.15.144:0/1060608055 conn(0x7ff15009aa00 msgr2=0x7ff15006fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:15.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 shutdown_connections 2024-03-08T08:27:15.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.859+0000 7ff155172640 1 -- 172.21.15.144:0/1060608055 wait complete. 2024-03-08T08:27:15.871 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 7475a6f5-267f-4139-83f9-568188376e6c 2 2024-03-08T08:27:15.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.970+0000 7f4d88f9a640 1 Processor -- start 2024-03-08T08:27:15.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.971+0000 7f4d88f9a640 1 -- start start 2024-03-08T08:27:15.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.971+0000 7f4d88f9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d8406edf0 0x7f4d8415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.971+0000 7f4d88f9a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d8415aeb0 0x7f4d8415c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.971+0000 7f4d88f9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8415ebb0 con 0x7f4d8415aad0 2024-03-08T08:27:15.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.971+0000 7f4d88f9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8415ee90 con 0x7f4d8406edf0 2024-03-08T08:27:15.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.971+0000 7f4d88f9a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8415f170 con 0x7f4d8415aeb0 2024-03-08T08:27:15.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d82d9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d8415aeb0 0x7f4d8415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d82d9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d8415aeb0 0x7f4d8415c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33618/0 (socket says 172.21.15.144:33618) 2024-03-08T08:27:15.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d82d9d640 1 -- 172.21.15.144:0/3123380733 learned_addr learned my addr 172.21.15.144:0/3123380733 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d81d9b640 1 --2- 172.21.15.144:0/3123380733 >> v2:172.21.15.105:3300/0 conn(0x7f4d8406edf0 0x7f4d8415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d82d9d640 1 -- 172.21.15.144:0/3123380733 >> v2:172.21.15.105:3300/0 conn(0x7f4d8406edf0 msgr2=0x7f4d8415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d82d9d640 1 --2- 172.21.15.144:0/3123380733 >> v2:172.21.15.105:3300/0 conn(0x7f4d8406edf0 0x7f4d8415e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d82d9d640 1 -- 172.21.15.144:0/3123380733 >> v1:172.21.15.105:6789/0 conn(0x7f4d8415aad0 legacy=0x7f4d8406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d82d9d640 1 -- 172.21.15.144:0/3123380733 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8415f450 con 0x7f4d8415aeb0 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.972+0000 7f4d81d9b640 1 --2- 172.21.15.144:0/3123380733 >> v2:172.21.15.105:3300/0 conn(0x7f4d8406edf0 0x7f4d8415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d82d9d640 1 --2- 172.21.15.144:0/3123380733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8415aeb0 0x7f4d8415c760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d84067a40 tx=0x7f4d7c02e180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44afb6ea4565eab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d8159a640 1 -- 172.21.15.144:0/3123380733 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00b0d0 con 0x7f4d8415aeb0 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d8159a640 1 -- 172.21.15.144:0/3123380733 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00b270 con 0x7f4d8415aeb0 2024-03-08T08:27:15.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d8159a640 1 -- 172.21.15.144:0/3123380733 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c009600 con 0x7f4d8415aeb0 2024-03-08T08:27:15.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3123380733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8415aeb0 msgr2=0x7f4d8415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d88f9a640 1 --2- 172.21.15.144:0/3123380733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8415aeb0 0x7f4d8415c760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4d84067a40 tx=0x7f4d7c02e180 comp rx=0 tx=0).stop 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3123380733 shutdown_connections 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d88f9a640 1 --2- 172.21.15.144:0/3123380733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8415aeb0 0x7f4d8415c760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.973+0000 7f4d88f9a640 1 --2- 172.21.15.144:0/3123380733 >> v2:172.21.15.105:3300/0 conn(0x7f4d8406edf0 0x7f4d8415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.974+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3123380733 >> 172.21.15.144:0/3123380733 conn(0x7f4d8406bd90 msgr2=0x7f4d8406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.974+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3123380733 shutdown_connections 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.974+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3123380733 wait complete. 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.975+0000 7f4d88f9a640 1 Processor -- start 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.975+0000 7f4d88f9a640 1 -- start start 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d88f9a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8406edf0 0x7f4d841549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d88f9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8415aad0 0x7f4d840b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d88f9a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d8415aeb0 0x7f4d84158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d88f9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840728a0 con 0x7f4d8415aad0 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d88f9a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8415b2d0 con 0x7f4d8406edf0 2024-03-08T08:27:15.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d88f9a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8415cca0 con 0x7f4d8415aeb0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d8259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8406edf0 0x7f4d841549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d82d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d8415aeb0 0x7f4d84158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d8259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8406edf0 0x7f4d841549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33622/0 (socket says 172.21.15.144:33622) 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d8259c640 1 -- 172.21.15.144:0/3307491674 learned_addr learned my addr 172.21.15.144:0/3307491674 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d81d9b640 1 --2- 172.21.15.144:0/3307491674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8415aad0 0x7f4d840b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d82d9d640 1 -- 172.21.15.144:0/3307491674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8406edf0 msgr2=0x7f4d841549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d82d9d640 1 --2- 172.21.15.144:0/3307491674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8406edf0 0x7f4d841549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d82d9d640 1 -- 172.21.15.144:0/3307491674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8415aad0 msgr2=0x7f4d840b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d82d9d640 1 --2- 172.21.15.144:0/3307491674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8415aad0 0x7f4d840b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d82d9d640 1 -- 172.21.15.144:0/3307491674 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d84158010 con 0x7f4d8415aeb0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d82d9d640 1 --2- 172.21.15.144:0/3307491674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d8415aeb0 0x7f4d84158d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c034000 tx=0x7f4d7c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d72ffd640 1 -- 172.21.15.144:0/3307491674 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c045020 con 0x7f4d8415aeb0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d72ffd640 1 -- 172.21.15.144:0/3307491674 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00b420 con 0x7f4d8415aeb0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d840b5590 con 0x7f4d8415aeb0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d72ffd640 1 -- 172.21.15.144:0/3307491674 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00b0d0 con 0x7f4d8415aeb0 2024-03-08T08:27:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.976+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d84158870 con 0x7f4d8415aeb0 2024-03-08T08:27:15.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.977+0000 7f4d72ffd640 1 -- 172.21.15.144:0/3307491674 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4d7c048020 con 0x7f4d8415aeb0 2024-03-08T08:27:15.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.977+0000 7f4d72ffd640 1 -- 172.21.15.144:0/3307491674 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4d7c040d20 con 0x7f4d8415aeb0 2024-03-08T08:27:15.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.977+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d841583c0 con 0x7f4d8415aeb0 2024-03-08T08:27:15.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:15.979+0000 7f4d72ffd640 1 -- 172.21.15.144:0/3307491674 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4d7c04f030 con 0x7f4d8415aeb0 2024-03-08T08:27:15.997 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:15.996+0000 7ff3f426c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T08:27:16.026 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.025+0000 7fdacde13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T08:27:16.052 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.052+0000 7ff3f426c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T08:27:16.053 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.052+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7475a6f5-267f-4139-83f9-568188376e6c", "id": 2} v 0) v1 -- 0x7f4d84075120 con 0x7f4d8415aeb0 2024-03-08T08:27:16.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.057+0000 7f4d72ffd640 1 -- 172.21.15.144:0/3307491674 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7475a6f5-267f-4139-83f9-568188376e6c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4d7c040450 con 0x7f4d8415aeb0 2024-03-08T08:27:16.058 INFO:teuthology.orchestra.run.smithi144.stdout:2 2024-03-08T08:27:16.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d8415aeb0 msgr2=0x7f4d84158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 --2- 172.21.15.144:0/3307491674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d8415aeb0 0x7f4d84158d90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c034000 tx=0x7f4d7c004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:16.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 shutdown_connections 2024-03-08T08:27:16.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 --2- 172.21.15.144:0/3307491674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d8415aeb0 0x7f4d84158d90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 --2- 172.21.15.144:0/3307491674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d8415aad0 0x7f4d840b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 --2- 172.21.15.144:0/3307491674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d8406edf0 0x7f4d841549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 >> 172.21.15.144:0/3307491674 conn(0x7f4d8406bd90 msgr2=0x7f4d8415d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 shutdown_connections 2024-03-08T08:27:16.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.058+0000 7f4d88f9a640 1 -- 172.21.15.144:0/3307491674 wait complete. 2024-03-08T08:27:16.070 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 68fae46e-aba8-42fe-a4db-693879d6aa8f 3 2024-03-08T08:27:16.085 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.083+0000 7fdacde13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T08:27:16.098 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.097+0000 7ff3f426c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T08:27:16.132 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.131+0000 7fdacde13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T08:27:16.151 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.150+0000 7ff3f426c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T08:27:16.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.169+0000 7fda874c3640 1 Processor -- start 2024-03-08T08:27:16.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.170+0000 7fda874c3640 1 -- start start 2024-03-08T08:27:16.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.170+0000 7fda874c3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.170+0000 7fda874c3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda8015df30 0x7fda800b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.170+0000 7fda874c3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fda800b5a60 0x7fda8009baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.170+0000 7fda874c3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8015f7b0 con 0x7fda800b5a60 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.170+0000 7fda874c3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8015e710 con 0x7fda8015df30 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.170+0000 7fda874c3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8015e9f0 con 0x7fda8015bea0 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda8525f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda84a5e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda8015df30 0x7fda800b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda84a5e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda8015df30 0x7fda800b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33634/0 (socket says 172.21.15.144:33634) 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda84a5e640 1 -- 172.21.15.144:0/3902243203 learned_addr learned my addr 172.21.15.144:0/3902243203 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda85a60640 1 --2- 172.21.15.144:0/3902243203 >> v2:172.21.15.105:3300/0 conn(0x7fda800b5a60 0x7fda8009baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda84a5e640 1 -- 172.21.15.144:0/3902243203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 msgr2=0x7fda800b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda84a5e640 1 --2- 172.21.15.144:0/3902243203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda84a5e640 1 -- 172.21.15.144:0/3902243203 >> v2:172.21.15.105:3300/0 conn(0x7fda800b5a60 msgr2=0x7fda8009baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:16.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda84a5e640 1 --2- 172.21.15.144:0/3902243203 >> v2:172.21.15.105:3300/0 conn(0x7fda800b5a60 0x7fda8009baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda84a5e640 1 -- 172.21.15.144:0/3902243203 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8015ecd0 con 0x7fda8015df30 2024-03-08T08:27:16.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.171+0000 7fda8525f640 1 --2- 172.21.15.144:0/3902243203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:16.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda84a5e640 1 --2- 172.21.15.144:0/3902243203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 0x7fda800b5520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fda800717e0 tx=0x7fda7002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=136bb168b8ea58ed server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:16.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda77fff640 1 -- 172.21.15.144:0/3902243203 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70035020 con 0x7fda8015df30 2024-03-08T08:27:16.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda77fff640 1 -- 172.21.15.144:0/3902243203 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda70031870 con 0x7fda8015df30 2024-03-08T08:27:16.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda77fff640 1 -- 172.21.15.144:0/3902243203 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70031c40 con 0x7fda8015df30 2024-03-08T08:27:16.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda874c3640 1 -- 172.21.15.144:0/3902243203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 msgr2=0x7fda800b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda874c3640 1 --2- 172.21.15.144:0/3902243203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 0x7fda800b5520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fda800717e0 tx=0x7fda7002b910 comp rx=0 tx=0).stop 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda874c3640 1 -- 172.21.15.144:0/3902243203 shutdown_connections 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda874c3640 1 --2- 172.21.15.144:0/3902243203 >> v2:172.21.15.105:3300/0 conn(0x7fda800b5a60 0x7fda8009baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda874c3640 1 --2- 172.21.15.144:0/3902243203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 0x7fda800b5520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda874c3640 1 --2- 172.21.15.144:0/3902243203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.172+0000 7fda874c3640 1 -- 172.21.15.144:0/3902243203 >> 172.21.15.144:0/3902243203 conn(0x7fda8009aa00 msgr2=0x7fda8009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.173+0000 7fda874c3640 1 -- 172.21.15.144:0/3902243203 shutdown_connections 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.173+0000 7fda874c3640 1 -- 172.21.15.144:0/3902243203 wait complete. 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.174+0000 7fda874c3640 1 Processor -- start 2024-03-08T08:27:16.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.174+0000 7fda874c3640 1 -- start start 2024-03-08T08:27:16.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.174+0000 7fda874c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda800b5a60 0x7fda8014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.174+0000 7fda874c3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.174+0000 7fda874c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 0x7fda80150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.174+0000 7fda874c3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8015e3b0 con 0x7fda800b5a60 2024-03-08T08:27:16.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.174+0000 7fda874c3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8015f290 con 0x7fda8015df30 2024-03-08T08:27:16.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.174+0000 7fda874c3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8009c030 con 0x7fda8015bea0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda85a60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 0x7fda80150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:39158/0 (socket says 172.21.15.144:39158) 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda85a60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 0x7fda80150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33642/0 (socket says 172.21.15.144:33642) 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 -- 172.21.15.144:0/1377290516 learned_addr learned my addr 172.21.15.144:0/1377290516 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda8525f640 1 --2- 172.21.15.144:0/1377290516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda800b5a60 0x7fda8014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 -- 172.21.15.144:0/1377290516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 msgr2=0x7fda80150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 --2- 172.21.15.144:0/1377290516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 0x7fda80150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 -- 172.21.15.144:0/1377290516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda800b5a60 msgr2=0x7fda8014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 --2- 172.21.15.144:0/1377290516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda800b5a60 0x7fda8014ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 -- 172.21.15.144:0/1377290516 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda800af140 con 0x7fda8015bea0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda84a5e640 1 --2- 172.21.15.144:0/1377290516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b29e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda75ffb640 1 -- 172.21.15.144:0/1377290516 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70031e10 con 0x7fda8015bea0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda75ffb640 1 -- 172.21.15.144:0/1377290516 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7003ace0 con 0x7fda8015bea0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8009ede0 con 0x7fda8015bea0 2024-03-08T08:27:16.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda75ffb640 1 -- 172.21.15.144:0/1377290516 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70044610 con 0x7fda8015bea0 2024-03-08T08:27:16.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.175+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8009f3a0 con 0x7fda8015bea0 2024-03-08T08:27:16.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.176+0000 7fda75ffb640 1 -- 172.21.15.144:0/1377290516 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fda7004b060 con 0x7fda8015bea0 2024-03-08T08:27:16.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.176+0000 7fda75ffb640 1 -- 172.21.15.144:0/1377290516 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fda7004dbf0 con 0x7fda8015bea0 2024-03-08T08:27:16.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.177+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda80075120 con 0x7fda8015bea0 2024-03-08T08:27:16.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.179+0000 7fda75ffb640 1 -- 172.21.15.144:0/1377290516 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fda70052020 con 0x7fda8015bea0 2024-03-08T08:27:16.188 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.187+0000 7fdacde13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T08:27:16.193 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.192+0000 7ff3f426c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T08:27:16.231 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.230+0000 7fdacde13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T08:27:16.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.250+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "68fae46e-aba8-42fe-a4db-693879d6aa8f", "id": 3} v 0) v1 -- 0x7fda8009bb80 con 0x7fda8015bea0 2024-03-08T08:27:16.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.255+0000 7fda75ffb640 1 -- 172.21.15.144:0/1377290516 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68fae46e-aba8-42fe-a4db-693879d6aa8f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fda7003f070 con 0x7fda8015bea0 2024-03-08T08:27:16.256 INFO:teuthology.orchestra.run.smithi144.stdout:3 2024-03-08T08:27:16.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 msgr2=0x7fda800b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 --2- 172.21.15.144:0/1377290516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b29e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda700046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 shutdown_connections 2024-03-08T08:27:16.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 --2- 172.21.15.144:0/1377290516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8015df30 0x7fda80150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 --2- 172.21.15.144:0/1377290516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8015bea0 0x7fda800b29e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 --2- 172.21.15.144:0/1377290516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda800b5a60 0x7fda8014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 >> 172.21.15.144:0/1377290516 conn(0x7fda8009aa00 msgr2=0x7fda8006e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 shutdown_connections 2024-03-08T08:27:16.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.256+0000 7fda874c3640 1 -- 172.21.15.144:0/1377290516 wait complete. 2024-03-08T08:27:16.268 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new c49af92b-b753-48dd-9ac9-11c95482035f 4 2024-03-08T08:27:16.308 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.307+0000 7ff3f426c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T08:27:16.351 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.350+0000 7fdacde13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T08:27:16.377 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.376+0000 7ff3f426c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T08:27:16.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.407+0000 7f9fbf381640 1 Processor -- start 2024-03-08T08:27:16.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.407+0000 7f9fbf381640 1 -- start start 2024-03-08T08:27:16.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.408+0000 7f9fbf381640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9fb806edf0 0x7f9fb815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.408+0000 7f9fbf381640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb815ebb0 con 0x7f9fb815aad0 2024-03-08T08:27:16.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.408+0000 7f9fbf381640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb815ee90 con 0x7f9fb815aeb0 2024-03-08T08:27:16.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.408+0000 7f9fbf381640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb815f170 con 0x7f9fb806edf0 2024-03-08T08:27:16.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.408+0000 7f9fbc91c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9fb806edf0 0x7f9fb815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.408+0000 7f9fbd91e640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9fb815aeb0 0x7f9fb815c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:45588/0 (socket says 172.21.15.144:45588) 2024-03-08T08:27:16.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.408+0000 7f9fbd91e640 1 -- 172.21.15.144:0/1563387179 learned_addr learned my addr 172.21.15.144:0/1563387179 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-03-08T08:27:16.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.409+0000 7f9fbc91c640 1 -- 172.21.15.144:0/1563387179 >> v1:172.21.15.144:6789/0 conn(0x7f9fb815aeb0 legacy=0x7f9fb815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.409+0000 7f9fbc91c640 1 -- 172.21.15.144:0/1563387179 >> v1:172.21.15.105:6789/0 conn(0x7f9fb815aad0 legacy=0x7f9fb806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.409+0000 7f9faffff640 1 -- 172.21.15.144:0/1563387179 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395421270 0 0) 0x7f9fb815ee90 con 0x7f9fb815aeb0 2024-03-08T08:27:16.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.409+0000 7f9fbc91c640 1 -- 172.21.15.144:0/1563387179 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb815f450 con 0x7f9fb806edf0 2024-03-08T08:27:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.409+0000 7f9fbc91c640 1 --2- 172.21.15.144:0/1563387179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb815e5d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400ba40 tx=0x7f9fb402e130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52fc4368136b0fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.409+0000 7f9faffff640 1 -- 172.21.15.144:0/1563387179 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb403b070 con 0x7f9fb806edf0 2024-03-08T08:27:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.409+0000 7f9faffff640 1 -- 172.21.15.144:0/1563387179 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb4036440 con 0x7f9fb806edf0 2024-03-08T08:27:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.409+0000 7f9faffff640 1 -- 172.21.15.144:0/1563387179 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4036810 con 0x7f9fb806edf0 2024-03-08T08:27:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.410+0000 7f9fbf381640 1 -- 172.21.15.144:0/1563387179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 msgr2=0x7f9fb815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.410+0000 7f9fbf381640 1 --2- 172.21.15.144:0/1563387179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb815e5d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400ba40 tx=0x7f9fb402e130 comp rx=0 tx=0).stop 2024-03-08T08:27:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.410+0000 7f9fbf381640 1 -- 172.21.15.144:0/1563387179 shutdown_connections 2024-03-08T08:27:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.410+0000 7f9fbf381640 1 --2- 172.21.15.144:0/1563387179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb815e5d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.410+0000 7f9fbf381640 1 -- 172.21.15.144:0/1563387179 >> 172.21.15.144:0/1563387179 conn(0x7f9fb806bd90 msgr2=0x7f9fb806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.410+0000 7f9fbf381640 1 -- 172.21.15.144:0/1563387179 shutdown_connections 2024-03-08T08:27:16.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.410+0000 7f9fbf381640 1 -- 172.21.15.144:0/1563387179 wait complete. 2024-03-08T08:27:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.411+0000 7f9fbf381640 1 Processor -- start 2024-03-08T08:27:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.411+0000 7f9fbf381640 1 -- start start 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbf381640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb813f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbf381640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fb815aad0 0x7f9fb8153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbf381640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb815aeb0 0x7f9fb8153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbf381640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80728a0 con 0x7f9fb815aeb0 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbf381640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb815b2d0 con 0x7f9fb806edf0 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbf381640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb815ce70 con 0x7f9fb815aad0 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd11d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb813f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd11d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb813f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33670/0 (socket says 172.21.15.144:33670) 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd11d640 1 -- 172.21.15.144:0/1224596436 learned_addr learned my addr 172.21.15.144:0/1224596436 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbc91c640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fb815aad0 0x7f9fb8153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd91e640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb815aeb0 0x7f9fb8153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd11d640 1 -- 172.21.15.144:0/1224596436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fb815aad0 msgr2=0x7f9fb8153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd11d640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fb815aad0 0x7f9fb8153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd11d640 1 -- 172.21.15.144:0/1224596436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb815aeb0 msgr2=0x7f9fb8153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd11d640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb815aeb0 0x7f9fb8153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.412+0000 7f9fbd11d640 1 -- 172.21.15.144:0/1224596436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb8158260 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.413+0000 7f9fbd11d640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb813f560 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800c450 tx=0x7f9fa800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.413+0000 7f9fadffb640 1 -- 172.21.15.144:0/1224596436 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80052d0 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.413+0000 7f9fadffb640 1 -- 172.21.15.144:0/1224596436 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8005470 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.413+0000 7f9fadffb640 1 -- 172.21.15.144:0/1224596436 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8012870 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.413+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb8158540 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.413+0000 7f9fadffb640 1 -- 172.21.15.144:0/1224596436 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9fa8012a10 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.413+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb80b4c10 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.413+0000 7f9fadffb640 1 -- 172.21.15.144:0/1224596436 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9fa8003b00 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.414+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb8075120 con 0x7f9fb806edf0 2024-03-08T08:27:16.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.416+0000 7f9fadffb640 1 -- 172.21.15.144:0/1224596436 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9fa80035f0 con 0x7f9fb806edf0 2024-03-08T08:27:16.426 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.423+0000 7ff3f426c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T08:27:16.427 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.424+0000 7fdacde13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T08:27:16.473 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.472+0000 7fdacde13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T08:27:16.475 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.475+0000 7ff3f426c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T08:27:16.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.489+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c49af92b-b753-48dd-9ac9-11c95482035f", "id": 4} v 0) v1 -- 0x7f9fb80673c0 con 0x7f9fb806edf0 2024-03-08T08:27:16.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.494+0000 7f9fadffb640 1 -- 172.21.15.144:0/1224596436 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c49af92b-b753-48dd-9ac9-11c95482035f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9fa80037d0 con 0x7f9fb806edf0 2024-03-08T08:27:16.495 INFO:teuthology.orchestra.run.smithi144.stdout:4 2024-03-08T08:27:16.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 msgr2=0x7f9fb813f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb813f560 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800c450 tx=0x7f9fa800c910 comp rx=0 tx=0).stop 2024-03-08T08:27:16.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 shutdown_connections 2024-03-08T08:27:16.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb815aeb0 0x7f9fb8153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fb815aad0 0x7f9fb8153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 --2- 172.21.15.144:0/1224596436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb806edf0 0x7f9fb813f560 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 >> 172.21.15.144:0/1224596436 conn(0x7f9fb806bd90 msgr2=0x7f9fb8154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 shutdown_connections 2024-03-08T08:27:16.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.495+0000 7f9fbf381640 1 -- 172.21.15.144:0/1224596436 wait complete. 2024-03-08T08:27:16.506 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 50a656b3-a9ed-4c0a-b80c-a6706559b239 5 2024-03-08T08:27:16.523 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.522+0000 7ff3f426c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T08:27:16.528 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.527+0000 7fdacde13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T08:27:16.578 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.577+0000 7fdacde13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T08:27:16.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.627+0000 7f2403abb640 1 Processor -- start 2024-03-08T08:27:16.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.627+0000 7f2403abb640 1 -- start start 2024-03-08T08:27:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2403abb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f23fc066990 0x7f23fc06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2403abb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc15e720 con 0x7f23fc0d07c0 2024-03-08T08:27:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2403abb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc066ec0 con 0x7f23fc15e340 2024-03-08T08:27:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2403abb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23fc15f020 con 0x7f23fc066990 2024-03-08T08:27:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2401857640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f23fc15e340 0x7f23fc0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:45596/0 (socket says 172.21.15.144:45596) 2024-03-08T08:27:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2401857640 1 -- 172.21.15.144:0/3751062147 learned_addr learned my addr 172.21.15.144:0/3751062147 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-03-08T08:27:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2402058640 1 --2- 172.21.15.144:0/3751062147 >> v2:172.21.15.105:3300/0 conn(0x7f23fc066990 0x7f23fc06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2400855640 1 -- 172.21.15.144:0/3751062147 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198487617 0 0) 0x7f23fc066ec0 con 0x7f23fc15e340 2024-03-08T08:27:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.628+0000 7f2400855640 1 -- 172.21.15.144:0/3751062147 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23e8003520 con 0x7f23fc15e340 2024-03-08T08:27:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.629+0000 7f2402058640 1 -- 172.21.15.144:0/3751062147 >> v1:172.21.15.144:6789/0 conn(0x7f23fc15e340 legacy=0x7f23fc0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.629+0000 7f2402058640 1 -- 172.21.15.144:0/3751062147 >> v1:172.21.15.105:6789/0 conn(0x7f23fc0d07c0 legacy=0x7f23fc0b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.629+0000 7f2400855640 1 -- 172.21.15.144:0/3751062147 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459579011 0 0) 0x7f23fc15e720 con 0x7f23fc0d07c0 2024-03-08T08:27:16.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.629+0000 7f2402058640 1 -- 172.21.15.144:0/3751062147 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc15e9d0 con 0x7f23fc066990 2024-03-08T08:27:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.629+0000 7f2402058640 1 --2- 172.21.15.144:0/3751062147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 0x7f23fc06b690 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009d90 tx=0x7f23f402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3b466ef42ff654 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2400855640 1 -- 172.21.15.144:0/3751062147 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4039070 con 0x7f23fc066990 2024-03-08T08:27:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2400855640 1 -- 172.21.15.144:0/3751062147 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f402cbd0 con 0x7f23fc066990 2024-03-08T08:27:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2400855640 1 -- 172.21.15.144:0/3751062147 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4034440 con 0x7f23fc066990 2024-03-08T08:27:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2403abb640 1 -- 172.21.15.144:0/3751062147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 msgr2=0x7f23fc06b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2403abb640 1 --2- 172.21.15.144:0/3751062147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 0x7f23fc06b690 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009d90 tx=0x7f23f402be20 comp rx=0 tx=0).stop 2024-03-08T08:27:16.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2403abb640 1 -- 172.21.15.144:0/3751062147 shutdown_connections 2024-03-08T08:27:16.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2403abb640 1 --2- 172.21.15.144:0/3751062147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 0x7f23fc06b690 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2403abb640 1 -- 172.21.15.144:0/3751062147 >> 172.21.15.144:0/3751062147 conn(0x7f23fc072110 msgr2=0x7f23fc072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2403abb640 1 -- 172.21.15.144:0/3751062147 shutdown_connections 2024-03-08T08:27:16.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.630+0000 7f2403abb640 1 -- 172.21.15.144:0/3751062147 wait complete. 2024-03-08T08:27:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.631+0000 7f2403abb640 1 Processor -- start 2024-03-08T08:27:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.631+0000 7f2403abb640 1 -- start start 2024-03-08T08:27:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2403abb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 0x7f23fc154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2403abb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23fc0d07c0 0x7f23fc1620f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2403abb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23fc15e340 0x7f23fc0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2403abb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc15f4b0 con 0x7f23fc066990 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2403abb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc0af530 con 0x7f23fc15e340 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2403abb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc161370 con 0x7f23fc0d07c0 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2402058640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23fc15e340 0x7f23fc0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2401056640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23fc0d07c0 0x7f23fc1620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2402058640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23fc15e340 0x7f23fc0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33682/0 (socket says 172.21.15.144:33682) 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2402058640 1 -- 172.21.15.144:0/1768341668 learned_addr learned my addr 172.21.15.144:0/1768341668 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.632+0000 7f2401857640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 0x7f23fc154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2402058640 1 -- 172.21.15.144:0/1768341668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23fc0d07c0 msgr2=0x7f23fc1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2402058640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23fc0d07c0 0x7f23fc1620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2402058640 1 -- 172.21.15.144:0/1768341668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 msgr2=0x7f23fc154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2402058640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 0x7f23fc154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2402058640 1 -- 172.21.15.144:0/1768341668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc155430 con 0x7f23fc15e340 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2401056640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23fc0d07c0 0x7f23fc1620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2402058640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23fc15e340 0x7f23fc0b13a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f23f4037040 tx=0x7f23f4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f23f27fc640 1 -- 172.21.15.144:0/1768341668 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4039050 con 0x7f23fc15e340 2024-03-08T08:27:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f23f27fc640 1 -- 172.21.15.144:0/1768341668 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f403a040 con 0x7f23fc15e340 2024-03-08T08:27:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f23f27fc640 1 -- 172.21.15.144:0/1768341668 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4003990 con 0x7f23fc15e340 2024-03-08T08:27:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc161610 con 0x7f23fc15e340 2024-03-08T08:27:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc161bd0 con 0x7f23fc15e340 2024-03-08T08:27:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f23f27fc640 1 -- 172.21.15.144:0/1768341668 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f23f4047020 con 0x7f23fc15e340 2024-03-08T08:27:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.633+0000 7f23f27fc640 1 -- 172.21.15.144:0/1768341668 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f23f4040a10 con 0x7f23fc15e340 2024-03-08T08:27:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.634+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23fc075120 con 0x7f23fc15e340 2024-03-08T08:27:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.636+0000 7f23f27fc640 1 -- 172.21.15.144:0/1768341668 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f23f4040cb0 con 0x7f23fc15e340 2024-03-08T08:27:16.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.708+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "50a656b3-a9ed-4c0a-b80c-a6706559b239", "id": 5} v 0) v1 -- 0x7f23fc072e90 con 0x7f23fc15e340 2024-03-08T08:27:16.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.713+0000 7f23f27fc640 1 -- 172.21.15.144:0/1768341668 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "50a656b3-a9ed-4c0a-b80c-a6706559b239", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f23f4040450 con 0x7f23fc15e340 2024-03-08T08:27:16.715 INFO:teuthology.orchestra.run.smithi144.stdout:5 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23fc15e340 msgr2=0x7f23fc0b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23fc15e340 0x7f23fc0b13a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f23f4037040 tx=0x7f23f4004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 shutdown_connections 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23fc15e340 0x7f23fc0b13a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23fc0d07c0 0x7f23fc1620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 --2- 172.21.15.144:0/1768341668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23fc066990 0x7f23fc154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 >> 172.21.15.144:0/1768341668 conn(0x7f23fc072110 msgr2=0x7f23fc06ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 shutdown_connections 2024-03-08T08:27:16.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.714+0000 7f2403abb640 1 -- 172.21.15.144:0/1768341668 wait complete. 2024-03-08T08:27:16.726 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 037ceb75-aa38-4cc5-a3ef-7a2e48d185fa 6 2024-03-08T08:27:16.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.826+0000 7f7acbf01640 1 Processor -- start 2024-03-08T08:27:16.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.826+0000 7f7acbf01640 1 -- start start 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7acbf01640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7ac406b390 0x7f7ac406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7acbf01640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7ac40b5990 0x7f7ac40ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7acbf01640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac406f830 con 0x7f7ac415df20 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7acbf01640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac406dbb0 con 0x7f7ac406b390 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7acbf01640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac406fad0 con 0x7f7ac40b5990 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7aca49e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7ac40b5990 0x7f7ac40ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7aca49e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7ac40b5990 0x7f7ac40ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33696/0 (socket says 172.21.15.144:33696) 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7aca49e640 1 -- 172.21.15.144:0/3188054847 learned_addr learned my addr 172.21.15.144:0/3188054847 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7ac949c640 1 --2- 172.21.15.144:0/3188054847 >> v2:172.21.15.105:3300/0 conn(0x7f7ac406b390 0x7f7ac406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7aca49e640 1 -- 172.21.15.144:0/3188054847 >> v2:172.21.15.105:3300/0 conn(0x7f7ac406b390 msgr2=0x7f7ac406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7aca49e640 1 --2- 172.21.15.144:0/3188054847 >> v2:172.21.15.105:3300/0 conn(0x7f7ac406b390 0x7f7ac406c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7aca49e640 1 -- 172.21.15.144:0/3188054847 >> v1:172.21.15.105:6789/0 conn(0x7f7ac415df20 legacy=0x7f7ac406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.827+0000 7f7aca49e640 1 -- 172.21.15.144:0/3188054847 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac406fdb0 con 0x7f7ac40b5990 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.828+0000 7f7aca49e640 1 --2- 172.21.15.144:0/3188054847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac40ad970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ac406f3a0 tx=0x7f7abc02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604cf255b641783d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.828+0000 7f7ac8c9b640 1 -- 172.21.15.144:0/3188054847 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc02c9f0 con 0x7f7ac40b5990 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.828+0000 7f7ac8c9b640 1 -- 172.21.15.144:0/3188054847 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc02cb90 con 0x7f7ac40b5990 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.828+0000 7f7ac8c9b640 1 -- 172.21.15.144:0/3188054847 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc0343f0 con 0x7f7ac40b5990 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.829+0000 7f7acbf01640 1 -- 172.21.15.144:0/3188054847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 msgr2=0x7f7ac40ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.829+0000 7f7acbf01640 1 --2- 172.21.15.144:0/3188054847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac40ad970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ac406f3a0 tx=0x7f7abc02be50 comp rx=0 tx=0).stop 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.829+0000 7f7acbf01640 1 -- 172.21.15.144:0/3188054847 shutdown_connections 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.829+0000 7f7acbf01640 1 --2- 172.21.15.144:0/3188054847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac40ad970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.829+0000 7f7acbf01640 1 --2- 172.21.15.144:0/3188054847 >> v2:172.21.15.105:3300/0 conn(0x7f7ac406b390 0x7f7ac406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.829+0000 7f7acbf01640 1 -- 172.21.15.144:0/3188054847 >> 172.21.15.144:0/3188054847 conn(0x7f7ac406bd90 msgr2=0x7f7ac406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.829+0000 7f7acbf01640 1 -- 172.21.15.144:0/3188054847 shutdown_connections 2024-03-08T08:27:16.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.829+0000 7f7acbf01640 1 -- 172.21.15.144:0/3188054847 wait complete. 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.830+0000 7f7acbf01640 1 Processor -- start 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.830+0000 7f7acbf01640 1 -- start start 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7acbf01640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac406b390 0x7f7ac4145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7acbf01640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac4160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7acbf01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ac415df20 0x7f7ac40afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7acbf01640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac40adeb0 con 0x7f7ac415df20 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7acbf01640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac406e0e0 con 0x7f7ac40b5990 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7acbf01640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac40ae030 con 0x7f7ac406b390 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac949c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac4160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac9c9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac406b390 0x7f7ac4145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac949c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac4160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33702/0 (socket says 172.21.15.144:33702) 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac949c640 1 -- 172.21.15.144:0/3861239563 learned_addr learned my addr 172.21.15.144:0/3861239563 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7aca49e640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ac415df20 0x7f7ac40afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac9c9d640 1 -- 172.21.15.144:0/3861239563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 msgr2=0x7f7ac4160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac9c9d640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac4160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac9c9d640 1 -- 172.21.15.144:0/3861239563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ac415df20 msgr2=0x7f7ac40afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac9c9d640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ac415df20 0x7f7ac40afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac9c9d640 1 -- 172.21.15.144:0/3861239563 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac41593a0 con 0x7f7ac406b390 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.831+0000 7f7ac949c640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac4160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.832+0000 7f7ac9c9d640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac406b390 0x7f7ac4145530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400b3e0 tx=0x7f7ab400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.832+0000 7f7aba7fc640 1 -- 172.21.15.144:0/3861239563 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4015030 con 0x7f7ac406b390 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.832+0000 7f7aba7fc640 1 -- 172.21.15.144:0/3861239563 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab4004020 con 0x7f7ac406b390 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.832+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac4159680 con 0x7f7ac406b390 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.832+0000 7f7aba7fc640 1 -- 172.21.15.144:0/3861239563 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4004450 con 0x7f7ac406b390 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.832+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac4159c40 con 0x7f7ac406b390 2024-03-08T08:27:16.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.832+0000 7f7aba7fc640 1 -- 172.21.15.144:0/3861239563 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7ab4018070 con 0x7f7ac406b390 2024-03-08T08:27:16.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.832+0000 7f7aba7fc640 1 -- 172.21.15.144:0/3861239563 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f7ab401a410 con 0x7f7ac406b390 2024-03-08T08:27:16.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.833+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac4075120 con 0x7f7ac406b390 2024-03-08T08:27:16.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.835+0000 7f7aba7fc640 1 -- 172.21.15.144:0/3861239563 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7ab40039b0 con 0x7f7ac406b390 2024-03-08T08:27:16.900 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:16.899+0000 7ff3f426c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T08:27:16.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.907+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "037ceb75-aa38-4cc5-a3ef-7a2e48d185fa", "id": 6} v 0) v1 -- 0x7f7ac415c5c0 con 0x7f7ac406b390 2024-03-08T08:27:16.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.912+0000 7f7aba7fc640 1 -- 172.21.15.144:0/3861239563 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "037ceb75-aa38-4cc5-a3ef-7a2e48d185fa", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7ab4003c60 con 0x7f7ac406b390 2024-03-08T08:27:16.914 INFO:teuthology.orchestra.run.smithi144.stdout:6 2024-03-08T08:27:16.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.913+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac406b390 msgr2=0x7f7ac4145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:16.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.913+0000 7f7acbf01640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac406b390 0x7f7ac4145530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400b3e0 tx=0x7f7ab400b8a0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.913+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 shutdown_connections 2024-03-08T08:27:16.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.913+0000 7f7acbf01640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ac415df20 0x7f7ac40afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.913+0000 7f7acbf01640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ac40b5990 0x7f7ac4160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.913+0000 7f7acbf01640 1 --2- 172.21.15.144:0/3861239563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ac406b390 0x7f7ac4145530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:16.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.913+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 >> 172.21.15.144:0/3861239563 conn(0x7f7ac406bd90 msgr2=0x7f7ac415cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:16.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.914+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 shutdown_connections 2024-03-08T08:27:16.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:16.914+0000 7f7acbf01640 1 -- 172.21.15.144:0/3861239563 wait complete. 2024-03-08T08:27:16.925 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 9feb8651-f7cd-474c-9b63-d51954617f4a 7 2024-03-08T08:27:16.979 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:16.978+0000 7fdacde13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T08:27:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.026+0000 7f542092c640 1 Processor -- start 2024-03-08T08:27:17.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.026+0000 7f542092c640 1 -- start start 2024-03-08T08:27:17.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f542092c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06ea60 0x7f541c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f542092c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0b4f50 con 0x7f541c0d01b0 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f542092c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c06b800 con 0x7f541c06dd00 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f542092c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c06ee40 con 0x7f541c06ea60 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f5419d9b640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f541c06dd00 0x7f541c06b020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:45600/0 (socket says 172.21.15.144:45600) 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f541a59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06ea60 0x7f541c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f5419d9b640 1 -- 172.21.15.144:0/2386085622 learned_addr learned my addr 172.21.15.144:0/2386085622 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f541959a640 1 -- 172.21.15.144:0/2386085622 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181022214 0 0) 0x7f541c06b800 con 0x7f541c06dd00 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f541a59c640 1 -- 172.21.15.144:0/2386085622 >> v1:172.21.15.144:6789/0 conn(0x7f541c06dd00 legacy=0x7f541c06b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f541a59c640 1 -- 172.21.15.144:0/2386085622 >> v1:172.21.15.105:6789/0 conn(0x7f541c0d01b0 legacy=0x7f541c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.027+0000 7f541a59c640 1 -- 172.21.15.144:0/2386085622 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c06bae0 con 0x7f541c06ea60 2024-03-08T08:27:17.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.028+0000 7f541a59c640 1 --2- 172.21.15.144:0/2386085622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06ea60 0x7f541c0cddf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097d0 tx=0x7f540c02cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80bdefb668c84e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:17.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.028+0000 7f541959a640 1 -- 172.21.15.144:0/2386085622 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c02e890 con 0x7f541c06ea60 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.028+0000 7f541959a640 1 -- 172.21.15.144:0/2386085622 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c02ea30 con 0x7f541c06ea60 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.028+0000 7f541959a640 1 -- 172.21.15.144:0/2386085622 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c02ee00 con 0x7f541c06ea60 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.029+0000 7f542092c640 1 -- 172.21.15.144:0/2386085622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06ea60 msgr2=0x7f541c0cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.029+0000 7f542092c640 1 --2- 172.21.15.144:0/2386085622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06ea60 0x7f541c0cddf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f540c0097d0 tx=0x7f540c02cad0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.029+0000 7f542092c640 1 -- 172.21.15.144:0/2386085622 shutdown_connections 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.029+0000 7f542092c640 1 --2- 172.21.15.144:0/2386085622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06ea60 0x7f541c0cddf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.029+0000 7f542092c640 1 -- 172.21.15.144:0/2386085622 >> 172.21.15.144:0/2386085622 conn(0x7f541c072110 msgr2=0x7f541c072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.029+0000 7f542092c640 1 -- 172.21.15.144:0/2386085622 shutdown_connections 2024-03-08T08:27:17.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.029+0000 7f542092c640 1 -- 172.21.15.144:0/2386085622 wait complete. 2024-03-08T08:27:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.030+0000 7f542092c640 1 Processor -- start 2024-03-08T08:27:17.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.030+0000 7f542092c640 1 -- start start 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f542092c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06dd00 0x7f541c097400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f542092c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f541c06ea60 0x7f541c1f0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f542092c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f541c0d01b0 0x7f541c1ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f542092c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0d0660 con 0x7f541c0d01b0 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f542092c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c06c0a0 con 0x7f541c06ea60 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f542092c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0d07e0 con 0x7f541c06dd00 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06dd00 0x7f541c097400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06dd00 0x7f541c097400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:39200/0 (socket says 172.21.15.144:39200) 2024-03-08T08:27:17.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 -- 172.21.15.144:0/4047121312 learned_addr learned my addr 172.21.15.144:0/4047121312 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f5419d9b640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f541c06ea60 0x7f541c1f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541ad9d640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f541c0d01b0 0x7f541c1ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 -- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f541c06ea60 msgr2=0x7f541c1f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f541c06ea60 0x7f541c1f0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 -- 172.21.15.144:0/4047121312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f541c0d01b0 msgr2=0x7f541c1ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f541c0d01b0 0x7f541c1ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 -- 172.21.15.144:0/4047121312 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c097940 con 0x7f541c06dd00 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f5419d9b640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f541c06ea60 0x7f541c1f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.031+0000 7f541a59c640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06dd00 0x7f541c097400 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f540c0098f0 tx=0x7f540c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.032+0000 7f540affd640 1 -- 172.21.15.144:0/4047121312 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c048070 con 0x7f541c06dd00 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.032+0000 7f540affd640 1 -- 172.21.15.144:0/4047121312 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c038400 con 0x7f541c06dd00 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.032+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c153f70 con 0x7f541c06dd00 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.032+0000 7f540affd640 1 -- 172.21.15.144:0/4047121312 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c0387f0 con 0x7f541c06dd00 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.032+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c1541f0 con 0x7f541c06dd00 2024-03-08T08:27:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.032+0000 7f540affd640 1 -- 172.21.15.144:0/4047121312 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f540c02e8c0 con 0x7f541c06dd00 2024-03-08T08:27:17.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.032+0000 7f540affd640 1 -- 172.21.15.144:0/4047121312 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f540c043a50 con 0x7f541c06dd00 2024-03-08T08:27:17.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.033+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c075120 con 0x7f541c06dd00 2024-03-08T08:27:17.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.035+0000 7f540affd640 1 -- 172.21.15.144:0/4047121312 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f541c153f70 con 0x7f541c06dd00 2024-03-08T08:27:17.036 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:17.036+0000 7ff3f426c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T08:27:17.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.107+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9feb8651-f7cd-474c-9b63-d51954617f4a", "id": 7} v 0) v1 -- 0x7f541c072e90 con 0x7f541c06dd00 2024-03-08T08:27:17.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.113+0000 7f540affd640 1 -- 172.21.15.144:0/4047121312 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9feb8651-f7cd-474c-9b63-d51954617f4a", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f540c034030 con 0x7f541c06dd00 2024-03-08T08:27:17.114 INFO:teuthology.orchestra.run.smithi144.stdout:7 2024-03-08T08:27:17.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06dd00 msgr2=0x7f541c097400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06dd00 0x7f541c097400 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f540c0098f0 tx=0x7f540c004730 comp rx=0 tx=0).stop 2024-03-08T08:27:17.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 shutdown_connections 2024-03-08T08:27:17.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f541c0d01b0 0x7f541c1ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f541c06ea60 0x7f541c1f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 --2- 172.21.15.144:0/4047121312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541c06dd00 0x7f541c097400 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 >> 172.21.15.144:0/4047121312 conn(0x7f541c072110 msgr2=0x7f541c0c7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:17.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 shutdown_connections 2024-03-08T08:27:17.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:27:17.114+0000 7f542092c640 1 -- 172.21.15.144:0/4047121312 wait complete. 2024-03-08T08:27:17.121 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:17.120+0000 7fdacde13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T08:27:17.125 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-08T08:27:17.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-08T08:27:17.128 INFO:tasks.ceph.osd.0:Started 2024-03-08T08:27:17.128 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-08T08:27:17.128 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-08T08:27:17.130 INFO:tasks.ceph.osd.1:Started 2024-03-08T08:27:17.130 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-08T08:27:17.130 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-08T08:27:17.132 INFO:tasks.ceph.osd.2:Started 2024-03-08T08:27:17.132 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-08T08:27:17.132 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-08T08:27:17.134 INFO:tasks.ceph.osd.3:Started 2024-03-08T08:27:17.134 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-08T08:27:17.135 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-08T08:27:17.137 INFO:tasks.ceph.osd.4:Started 2024-03-08T08:27:17.137 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-08T08:27:17.137 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-08T08:27:17.139 INFO:tasks.ceph.osd.5:Started 2024-03-08T08:27:17.139 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-08T08:27:17.139 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-08T08:27:17.141 INFO:tasks.ceph.osd.6:Started 2024-03-08T08:27:17.141 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-08T08:27:17.141 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-08T08:27:17.144 INFO:tasks.ceph.osd.7:Started 2024-03-08T08:27:17.144 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:17.150 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:17.149+0000 7ff3f426c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T08:27:17.225 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:17.224+0000 7ff3f426c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T08:27:17.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.255+0000 7facb0fc7640 1 Processor -- start 2024-03-08T08:27:17.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.256+0000 7facb0fc7640 1 -- start start 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facb0fc7640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7facac06edf0 0x7facac15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facb0fc7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facb0fc7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facac15ebb0 con 0x7facac15aad0 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facb0fc7640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facac15ee90 con 0x7facac06edf0 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facb0fc7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac15f170 con 0x7facac15aeb0 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facaad9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facaad9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac15c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:50756/0 (socket says 172.21.15.105:50756) 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facaad9d640 1 -- 172.21.15.105:0/1038827050 learned_addr learned my addr 172.21.15.105:0/1038827050 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:17.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facaa59c640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7facac15aad0 0x7facac06e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:54708/0 (socket says 172.21.15.105:54708) 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7faca9d9b640 1 --2- 172.21.15.105:0/1038827050 >> v2:172.21.15.144:3300/0 conn(0x7facac06edf0 0x7facac15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7faca9d9b640 1 -- 172.21.15.105:0/1038827050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 msgr2=0x7facac15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7faca9d9b640 1 --2- 172.21.15.105:0/1038827050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7faca9d9b640 1 -- 172.21.15.105:0/1038827050 >> v1:172.21.15.144:6789/0 conn(0x7facac15aad0 legacy=0x7facac06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7faca9d9b640 1 -- 172.21.15.105:0/1038827050 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac15f450 con 0x7facac06edf0 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.257+0000 7facaad9d640 1 --2- 172.21.15.105:0/1038827050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.258+0000 7faca9d9b640 1 --2- 172.21.15.105:0/1038827050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac06edf0 0x7facac15e5d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0097a0 tx=0x7fac9c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b26ceaa7522414 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.258+0000 7faca959a640 1 -- 172.21.15.105:0/1038827050 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c035020 con 0x7facac06edf0 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.258+0000 7faca959a640 1 -- 172.21.15.105:0/1038827050 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c031830 con 0x7facac06edf0 2024-03-08T08:27:17.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.258+0000 7faca959a640 1 -- 172.21.15.105:0/1038827050 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c031c00 con 0x7facac06edf0 2024-03-08T08:27:17.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.259+0000 7facb0fc7640 1 -- 172.21.15.105:0/1038827050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac06edf0 msgr2=0x7facac15e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.259+0000 7facb0fc7640 1 --2- 172.21.15.105:0/1038827050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac06edf0 0x7facac15e5d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0097a0 tx=0x7fac9c02b8d0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.259+0000 7facb0fc7640 1 -- 172.21.15.105:0/1038827050 shutdown_connections 2024-03-08T08:27:17.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.259+0000 7facb0fc7640 1 --2- 172.21.15.105:0/1038827050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.259+0000 7facb0fc7640 1 --2- 172.21.15.105:0/1038827050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac06edf0 0x7facac15e5d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.259+0000 7facb0fc7640 1 -- 172.21.15.105:0/1038827050 >> 172.21.15.105:0/1038827050 conn(0x7facac06bd90 msgr2=0x7facac06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.259+0000 7facb0fc7640 1 -- 172.21.15.105:0/1038827050 shutdown_connections 2024-03-08T08:27:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.259+0000 7facb0fc7640 1 -- 172.21.15.105:0/1038827050 wait complete. 2024-03-08T08:27:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.260+0000 7facb0fc7640 1 Processor -- start 2024-03-08T08:27:17.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.260+0000 7facb0fc7640 1 -- start start 2024-03-08T08:27:17.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facb0fc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facac06edf0 0x7facac154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facb0fc7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15aad0 0x7facac156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facb0fc7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:17.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facb0fc7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac0728a0 con 0x7facac06edf0 2024-03-08T08:27:17.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facac06edf0 0x7facac154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facb0fc7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac15b320 con 0x7facac15aad0 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facb0fc7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac15cca0 con 0x7facac15aeb0 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facac06edf0 0x7facac154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35970/0 (socket says 172.21.15.105:35970) 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaa59c640 1 -- 172.21.15.105:0/536572441 learned_addr learned my addr 172.21.15.105:0/536572441 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7faca9d9b640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15aad0 0x7facac156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaad9d640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaa59c640 1 -- 172.21.15.105:0/536572441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 msgr2=0x7facac0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaa59c640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaa59c640 1 -- 172.21.15.105:0/536572441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15aad0 msgr2=0x7facac156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaa59c640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15aad0 0x7facac156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.261+0000 7facaa59c640 1 -- 172.21.15.105:0/536572441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac0a9680 con 0x7facac06edf0 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.262+0000 7facaa59c640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facac06edf0 0x7facac154c20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c450 tx=0x7fac9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.262+0000 7faca2ffd640 1 -- 172.21.15.105:0/536572441 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94018070 con 0x7facac06edf0 2024-03-08T08:27:17.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.262+0000 7faca2ffd640 1 -- 172.21.15.105:0/536572441 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac940074a0 con 0x7facac06edf0 2024-03-08T08:27:17.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.262+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac157420 con 0x7facac06edf0 2024-03-08T08:27:17.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.262+0000 7faca2ffd640 1 -- 172.21.15.105:0/536572441 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94004a20 con 0x7facac06edf0 2024-03-08T08:27:17.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.262+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac1579e0 con 0x7facac06edf0 2024-03-08T08:27:17.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.262+0000 7faca2ffd640 1 -- 172.21.15.105:0/536572441 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fac9401b070 con 0x7facac06edf0 2024-03-08T08:27:17.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.262+0000 7faca2ffd640 1 -- 172.21.15.105:0/536572441 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fac94012b10 con 0x7facac06edf0 2024-03-08T08:27:17.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.263+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac06fcb0 con 0x7facac06edf0 2024-03-08T08:27:17.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.267+0000 7faca2ffd640 1 -- 172.21.15.105:0/536572441 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7facac06fcb0 con 0x7facac06edf0 2024-03-08T08:27:17.273 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:17.272+0000 7fdacde13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T08:27:17.319 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:17.317+0000 7fdacde13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T08:27:17.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.328+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facac072b80 con 0x7facac06edf0 2024-03-08T08:27:17.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.328+0000 7faca2ffd640 1 -- 172.21.15.105:0/536572441 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7facac072b80 con 0x7facac06edf0 2024-03-08T08:27:17.329 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:17.329 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:17.109283+0000","last_up_change":"0.000000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:17.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facac06edf0 msgr2=0x7facac154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:17.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facac06edf0 0x7facac154c20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c450 tx=0x7fac9400c910 comp rx=0 tx=0).stop 2024-03-08T08:27:17.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 shutdown_connections 2024-03-08T08:27:17.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facac15aeb0 0x7facac0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facac15aad0 0x7facac156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 --2- 172.21.15.105:0/536572441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facac06edf0 0x7facac154c20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:17.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 >> 172.21.15.105:0/536572441 conn(0x7facac06bd90 msgr2=0x7facac06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:17.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 shutdown_connections 2024-03-08T08:27:17.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:17.329+0000 7facb0fc7640 1 -- 172.21.15.105:0/536572441 wait complete. 2024-03-08T08:27:17.341 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-08T08:27:17.341 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-08T08:27:17.346 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:17.345+0000 7ff3f426c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T08:27:17.430 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:17.429+0000 7fdacde13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T08:27:17.441 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:17.440+0000 7ff3f426c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T08:27:17.526 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:17.525+0000 7fdacde13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T08:27:17.598 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:17.597+0000 7ff3f426c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T08:27:17.640 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:17.639+0000 7ff3f426c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T08:27:17.680 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:17.679+0000 7fdacde13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T08:27:17.722 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:17.721+0000 7fdacde13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T08:27:17.750 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-03-08T08:27:17.749+0000 7f7c1fc06640 -1 Falling back to public interface 2024-03-08T08:27:17.750 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-03-08T08:27:17.749+0000 7f08a7a19640 -1 Falling back to public interface 2024-03-08T08:27:17.750 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-03-08T08:27:17.749+0000 7f7c1ebb5640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-03-08T08:27:17.751 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-03-08T08:27:17.749+0000 7f08a6496640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6802/0: (98) Address already in use 2024-03-08T08:27:17.755 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-03-08T08:27:17.754+0000 7fadf0437640 -1 Falling back to public interface 2024-03-08T08:27:17.760 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-03-08T08:27:17.759+0000 7f18cc68b640 -1 Falling back to public interface 2024-03-08T08:27:17.765 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-03-08T08:27:17.765+0000 7f7c0200f640 -1 Falling back to public interface 2024-03-08T08:27:17.766 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-03-08T08:27:17.765+0000 7f87141bd640 -1 Falling back to public interface 2024-03-08T08:27:17.768 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-03-08T08:27:17.767+0000 7f5048e24640 -1 Falling back to public interface 2024-03-08T08:27:17.772 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-03-08T08:27:17.771+0000 7f44c04b1640 -1 Falling back to public interface 2024-03-08T08:27:17.796 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:17.796+0000 7ff3f426c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T08:27:17.883 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:17.882+0000 7fdacde13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T08:27:18.146 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:18.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.237+0000 7f7a7862e640 1 Processor -- start 2024-03-08T08:27:18.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.237+0000 7f7a7862e640 1 -- start start 2024-03-08T08:27:18.245 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a7862e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a700616a0 0x7f7a7015ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:18.245 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a7862e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7015c0c0 con 0x7f7a7015acd0 2024-03-08T08:27:18.245 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a7862e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a700af1f0 con 0x7f7a7006aff0 2024-03-08T08:27:18.245 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a7862e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700af4d0 con 0x7f7a700616a0 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a763ca640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7a7015acd0 0x7f7a7015e7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37082/0 (socket says 172.21.15.105:37082) 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a763ca640 1 -- 172.21.15.105:0/3427553721 learned_addr learned my addr 172.21.15.105:0/3427553721 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a75bc9640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a700616a0 0x7f7a7015ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571863989 0 0) 0x7f7a6c001110 con 0x7f7a7015acd0 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a5c005730 con 0x7f7a7015acd0 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935996669 0 0) 0x7f7a700af1f0 con 0x7f7a7006aff0 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a6c001110 con 0x7f7a7006aff0 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663036935 0 0) 0x7f7a5c005730 con 0x7f7a7015acd0 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a700616a0 msgr2=0x7f7a7015ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a700616a0 0x7f7a7015ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:18.246 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 >> v1:172.21.15.144:6789/0 conn(0x7f7a7006aff0 legacy=0x7f7a7006b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700af7b0 con 0x7f7a7015acd0 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f7a6c001840 con 0x7f7a7015acd0 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 >> v1:172.21.15.105:6789/0 conn(0x7f7a7015acd0 legacy=0x7f7a7015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c005d80 0x7f7a5c006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a700616a0 0x7f7a5c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a5c006f50 0x7f7a5c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700af4d0 con 0x7f7a5c006f50 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c00dd60 con 0x7f7a5c005d80 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.238+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c00df30 con 0x7f7a700616a0 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a76bcb640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a5c006f50 0x7f7a5c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a763ca640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c005d80 0x7f7a5c006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:18.247 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a75bc9640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a700616a0 0x7f7a5c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a76bcb640 1 -- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a700616a0 msgr2=0x7f7a5c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a76bcb640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a700616a0 0x7f7a5c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a76bcb640 1 -- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c005d80 msgr2=0x7f7a5c006180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a76bcb640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c005d80 0x7f7a5c006180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a76bcb640 1 -- 172.21.15.105:0/3427553721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c005730 con 0x7f7a5c006f50 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.239+0000 7f7a76bcb640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a5c006f50 0x7f7a5c0073d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a68002000 tx=0x7f7a5c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b390e1f08b822666 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68001bf0 con 0x7f7a5c006f50 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68003b70 con 0x7f7a5c006f50 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a753c8640 1 -- 172.21.15.105:0/3427553721 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680050d0 con 0x7f7a5c006f50 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a7862e640 1 -- 172.21.15.105:0/3427553721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a5c006f50 msgr2=0x7f7a5c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a7862e640 1 --2- 172.21.15.105:0/3427553721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a5c006f50 0x7f7a5c0073d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a68002000 tx=0x7f7a5c007a40 comp rx=0 tx=0).stop 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a76bcb640 1 -- 172.21.15.105:0/3427553721 reap_dead start 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a7862e640 1 -- 172.21.15.105:0/3427553721 shutdown_connections 2024-03-08T08:27:18.248 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a7862e640 1 -- 172.21.15.105:0/3427553721 >> 172.21.15.105:0/3427553721 conn(0x7f7a700b5110 msgr2=0x7f7a700b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a7862e640 1 -- 172.21.15.105:0/3427553721 shutdown_connections 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.240+0000 7f7a7862e640 1 -- 172.21.15.105:0/3427553721 wait complete. 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.241+0000 7f7a7862e640 1 Processor -- start 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.241+0000 7f7a7862e640 1 -- start start 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a7862e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c006f50 0x7f7a70151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a7862e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a700b2960 0x7f7a701f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a7862e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a701f79f0 0x7f7a701f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a7862e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7015ef00 con 0x7f7a701f79f0 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a7862e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a70061b00 con 0x7f7a700b2960 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a7862e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7015f080 con 0x7f7a5c006f50 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a76bcb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a701f79f0 0x7f7a701f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a76bcb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a701f79f0 0x7f7a701f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35984/0 (socket says 172.21.15.105:35984) 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a76bcb640 1 -- 172.21.15.105:0/3023126068 learned_addr learned my addr 172.21.15.105:0/3023126068 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:18.249 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a763ca640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c006f50 0x7f7a70151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.242+0000 7f7a75bc9640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a700b2960 0x7f7a701f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a76bcb640 1 -- 172.21.15.105:0/3023126068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c006f50 msgr2=0x7f7a70151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a76bcb640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c006f50 0x7f7a70151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a76bcb640 1 -- 172.21.15.105:0/3023126068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a700b2960 msgr2=0x7f7a701f74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a76bcb640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a700b2960 0x7f7a701f74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a76bcb640 1 -- 172.21.15.105:0/3023126068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700ce970 con 0x7f7a701f79f0 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a76bcb640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a701f79f0 0x7f7a701f7e70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a68002000 tx=0x7f7a68001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a66ffd640 1 -- 172.21.15.105:0/3023126068 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68042070 con 0x7f7a701f79f0 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a66ffd640 1 -- 172.21.15.105:0/3023126068 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68005380 con 0x7f7a701f79f0 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a700cec50 con 0x7f7a701f79f0 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a66ffd640 1 -- 172.21.15.105:0/3023126068 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68039040 con 0x7f7a701f79f0 2024-03-08T08:27:18.250 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a700cf210 con 0x7f7a701f79f0 2024-03-08T08:27:18.251 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a66ffd640 1 -- 172.21.15.105:0/3023126068 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7a68045070 con 0x7f7a701f79f0 2024-03-08T08:27:18.251 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.243+0000 7f7a66ffd640 1 -- 172.21.15.105:0/3023126068 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7a6803db00 con 0x7f7a701f79f0 2024-03-08T08:27:18.251 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.244+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a700caa50 con 0x7f7a701f79f0 2024-03-08T08:27:18.251 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.246+0000 7f7a66ffd640 1 -- 172.21.15.105:0/3023126068 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7a700caa50 con 0x7f7a701f79f0 2024-03-08T08:27:18.307 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.307+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a700b2d40 con 0x7f7a701f79f0 2024-03-08T08:27:18.308 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.307+0000 7f7a66ffd640 1 -- 172.21.15.105:0/3023126068 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f7a68005750 con 0x7f7a701f79f0 2024-03-08T08:27:18.308 INFO:teuthology.misc.health.smithi105.stdout: 2024-03-08T08:27:18.308 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":10,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:17.109283+0000","last_up_change":"0.000000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:18.309 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a701f79f0 msgr2=0x7f7a701f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:18.309 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a701f79f0 0x7f7a701f7e70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a68002000 tx=0x7f7a68001650 comp rx=0 tx=0).stop 2024-03-08T08:27:18.310 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 shutdown_connections 2024-03-08T08:27:18.310 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a701f79f0 0x7f7a701f7e70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:18.310 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a700b2960 0x7f7a701f74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:18.310 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 --2- 172.21.15.105:0/3023126068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c006f50 0x7f7a70151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:18.310 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 >> 172.21.15.105:0/3023126068 conn(0x7f7a700b5110 msgr2=0x7f7a701552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:18.310 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 shutdown_connections 2024-03-08T08:27:18.310 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:18.308+0000 7f7a7862e640 1 -- 172.21.15.105:0/3023126068 wait complete. 2024-03-08T08:27:18.319 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-08T08:27:18.629 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:18.629+0000 7ff3f426c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T08:27:18.690 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:18.689+0000 7fdacde13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T08:27:18.907 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:18.906+0000 7ff3f426c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T08:27:18.959 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:18.958+0000 7ff3f426c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T08:27:18.965 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:18.964+0000 7fdacde13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T08:27:19.015 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:19.014+0000 7fdacde13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T08:27:19.075 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:19.074+0000 7ff3f426c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T08:27:19.133 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-03-08T08:27:19.132+0000 7fdacde13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T08:27:20.100 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:20.099+0000 7ff3956d1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-08T08:27:20.102 INFO:tasks.ceph.mgr.y.smithi105.stderr:2024-03-08T08:27:20.101+0000 7ff3956d1640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-08T08:27:21.251 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-03-08T08:27:21.250+0000 7f7c0200f640 -1 osd.3 0 log_to_monitors true 2024-03-08T08:27:21.273 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-03-08T08:27:21.272+0000 7fadf0437640 -1 osd.5 0 log_to_monitors true 2024-03-08T08:27:21.273 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-03-08T08:27:21.272+0000 7f08a7a19640 -1 osd.6 0 log_to_monitors true 2024-03-08T08:27:21.279 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-03-08T08:27:21.278+0000 7f5048e24640 -1 osd.7 0 log_to_monitors true 2024-03-08T08:27:21.280 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-03-08T08:27:21.279+0000 7f7c1fc06640 -1 osd.4 0 log_to_monitors true 2024-03-08T08:27:21.295 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-03-08T08:27:21.295+0000 7f18cc68b640 -1 osd.2 0 log_to_monitors true 2024-03-08T08:27:21.322 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-03-08T08:27:21.322+0000 7f44c04b1640 -1 osd.1 0 log_to_monitors true 2024-03-08T08:27:21.338 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-03-08T08:27:21.337+0000 7f87141bd640 -1 osd.0 0 log_to_monitors true 2024-03-08T08:27:25.033 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-03-08T08:27:25.032+0000 7f18c7dfe640 -1 osd.2 0 waiting for initial osdmap 2024-03-08T08:27:25.044 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-03-08T08:27:25.044+0000 7f18c0430640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T08:27:25.124 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:25.158 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-03-08T08:27:25.157+0000 7f7bfd782640 -1 osd.3 0 waiting for initial osdmap 2024-03-08T08:27:25.179 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-03-08T08:27:25.178+0000 7f7bf5db4640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T08:27:25.179 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-03-08T08:27:25.178+0000 7f44bd45c640 -1 osd.1 0 waiting for initial osdmap 2024-03-08T08:27:25.190 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-03-08T08:27:25.190+0000 7f44b4256640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T08:27:25.195 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-03-08T08:27:25.194+0000 7f08a2c5a640 -1 osd.6 0 waiting for initial osdmap 2024-03-08T08:27:25.198 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-03-08T08:27:25.198+0000 7f870f930640 -1 osd.0 0 waiting for initial osdmap 2024-03-08T08:27:25.206 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-03-08T08:27:25.203+0000 7fadebbaa640 -1 osd.5 0 waiting for initial osdmap 2024-03-08T08:27:25.211 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-03-08T08:27:25.210+0000 7f089b28c640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T08:27:25.215 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-03-08T08:27:25.215+0000 7f8707f62640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T08:27:25.216 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-03-08T08:27:25.215+0000 7fade41dc640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T08:27:25.221 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-03-08T08:27:25.220+0000 7f7c1b379640 -1 osd.4 0 waiting for initial osdmap 2024-03-08T08:27:25.232 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.231+0000 7fde21b3e640 1 Processor -- start 2024-03-08T08:27:25.232 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.231+0000 7fde21b3e640 1 -- start start 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde21b3e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c155280 0x7fde1c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde21b3e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fde1c0b1b90 0x7fde1c0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde21b3e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c15c1f0 con 0x7fde1c0b0820 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde21b3e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c0b0c90 con 0x7fde1c0b1b90 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde21b3e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b2c20 con 0x7fde1c155280 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde13fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fde1c0b1b90 0x7fde1c0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde1bfff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fde1c0b0820 0x7fde1c15e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:60280/0 (socket says 172.21.15.105:60280) 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde13fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fde1c0b1b90 0x7fde1c0b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53474/0 (socket says 172.21.15.105:53474) 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde1b7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c155280 0x7fde1c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:25.234 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.232+0000 7fde1bfff640 1 -- 172.21.15.105:0/1061339328 learned_addr learned my addr 172.21.15.105:0/1061339328 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:25.235 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.233+0000 7fde1b7fe640 1 -- 172.21.15.105:0/1061339328 >> v2:172.21.15.144:3300/0 conn(0x7fde1c0b1b90 msgr2=0x7fde1c0b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:25.235 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.233+0000 7fde1b7fe640 1 --2- 172.21.15.105:0/1061339328 >> v2:172.21.15.144:3300/0 conn(0x7fde1c0b1b90 0x7fde1c0b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.235 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.233+0000 7fde1b7fe640 1 -- 172.21.15.105:0/1061339328 >> v1:172.21.15.144:6789/0 conn(0x7fde1c0b0820 legacy=0x7fde1c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:25.235 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.233+0000 7fde1b7fe640 1 -- 172.21.15.105:0/1061339328 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0b2010 con 0x7fde1c155280 2024-03-08T08:27:25.235 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.233+0000 7fde13fff640 1 --2- 172.21.15.105:0/1061339328 >> v2:172.21.15.144:3300/0 conn(0x7fde1c0b1b90 0x7fde1c0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:25.235 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-03-08T08:27:25.234+0000 7f7c139ab640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.233+0000 7fde1b7fe640 1 --2- 172.21.15.105:0/1061339328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c155280 0x7fde1c15bcb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0097a0 tx=0x7fde0c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc77087ca3cd192 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde1affd640 1 -- 172.21.15.105:0/1061339328 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c0315a0 con 0x7fde1c155280 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde1affd640 1 -- 172.21.15.105:0/1061339328 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c037070 con 0x7fde1c155280 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde1affd640 1 -- 172.21.15.105:0/1061339328 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c031960 con 0x7fde1c155280 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde21b3e640 1 -- 172.21.15.105:0/1061339328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c155280 msgr2=0x7fde1c15bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde21b3e640 1 --2- 172.21.15.105:0/1061339328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c155280 0x7fde1c15bcb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0097a0 tx=0x7fde0c02b840 comp rx=0 tx=0).stop 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde21b3e640 1 -- 172.21.15.105:0/1061339328 shutdown_connections 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde21b3e640 1 --2- 172.21.15.105:0/1061339328 >> v2:172.21.15.144:3300/0 conn(0x7fde1c0b1b90 0x7fde1c0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.238 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde21b3e640 1 --2- 172.21.15.105:0/1061339328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c155280 0x7fde1c15bcb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde21b3e640 1 -- 172.21.15.105:0/1061339328 >> 172.21.15.105:0/1061339328 conn(0x7fde1c0712f0 msgr2=0x7fde1c0b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde21b3e640 1 -- 172.21.15.105:0/1061339328 shutdown_connections 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.234+0000 7fde21b3e640 1 -- 172.21.15.105:0/1061339328 wait complete. 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.235+0000 7fde21b3e640 1 Processor -- start 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.235+0000 7fde21b3e640 1 -- start start 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde21b3e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c0b0820 0x7fde1c151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde21b3e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1c0b1b90 0x7fde1c0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde21b3e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde1c155280 0x7fde1c0adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde21b3e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b28b0 con 0x7fde1c0b1b90 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde21b3e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b25d0 con 0x7fde1c155280 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde21b3e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c155660 con 0x7fde1c0b0820 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1c0b1b90 0x7fde1c0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1c0b1b90 0x7fde1c0ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51610/0 (socket says 172.21.15.105:51610) 2024-03-08T08:27:25.239 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde13fff640 1 -- 172.21.15.105:0/2442003367 learned_addr learned my addr 172.21.15.105:0/2442003367 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde1bfff640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde1c155280 0x7fde1c0adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde1b7fe640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c0b0820 0x7fde1c151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde13fff640 1 -- 172.21.15.105:0/2442003367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c0b0820 msgr2=0x7fde1c151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde13fff640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c0b0820 0x7fde1c151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde13fff640 1 -- 172.21.15.105:0/2442003367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde1c155280 msgr2=0x7fde1c0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde13fff640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde1c155280 0x7fde1c0adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.236+0000 7fde13fff640 1 -- 172.21.15.105:0/2442003367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0ae1f0 con 0x7fde1c0b1b90 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.237+0000 7fde13fff640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1c0b1b90 0x7fde1c0ad680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e9b0 tx=0x7fde0400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.237+0000 7fde18ff9640 1 -- 172.21.15.105:0/2442003367 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04017070 con 0x7fde1c0b1b90 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.237+0000 7fde18ff9640 1 -- 172.21.15.105:0/2442003367 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0400c9a0 con 0x7fde1c0b1b90 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.237+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c0ae4d0 con 0x7fde1c0b1b90 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.237+0000 7fde18ff9640 1 -- 172.21.15.105:0/2442003367 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0400cd50 con 0x7fde1c0b1b90 2024-03-08T08:27:25.240 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.237+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c0aea90 con 0x7fde1c0b1b90 2024-03-08T08:27:25.241 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.238+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c06a380 con 0x7fde1c0b1b90 2024-03-08T08:27:25.241 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.238+0000 7fde18ff9640 1 -- 172.21.15.105:0/2442003367 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fde0401a070 con 0x7fde1c0b1b90 2024-03-08T08:27:25.241 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.238+0000 7fde18ff9640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fddfc06b440 0x7fddfc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:25.241 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.238+0000 7fde1b7fe640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fddfc06b440 0x7fddfc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:25.242 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.238+0000 7fde18ff9640 1 -- 172.21.15.105:0/2442003367 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fde04089190 con 0x7fde1c0b1b90 2024-03-08T08:27:25.242 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.239+0000 7fde1b7fe640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fddfc06b440 0x7fddfc06d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde0c036000 tx=0x7fde0c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:25.242 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-03-08T08:27:25.240+0000 7f5045dcf640 -1 osd.7 0 waiting for initial osdmap 2024-03-08T08:27:25.244 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.243+0000 7fde18ff9640 1 -- 172.21.15.105:0/2442003367 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde04058830 con 0x7fde1c0b1b90 2024-03-08T08:27:25.257 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-03-08T08:27:25.256+0000 7f503cbc9640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T08:27:25.419 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.418+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde1c071910 con 0x7fde1c0b1b90 2024-03-08T08:27:25.420 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.419+0000 7fde18ff9640 1 -- 172.21.15.105:0/2442003367 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fde0405c8d0 con 0x7fde1c0b1b90 2024-03-08T08:27:25.420 INFO:teuthology.misc.health.smithi105.stdout: 2024-03-08T08:27:25.420 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":12,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:23.100700+0000","last_up_change":"0.000000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:25.422 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fddfc06b440 msgr2=0x7fddfc06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:25.422 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fddfc06b440 0x7fddfc06d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde0c036000 tx=0x7fde0c03d040 comp rx=0 tx=0).stop 2024-03-08T08:27:25.422 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1c0b1b90 msgr2=0x7fde1c0ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:25.422 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1c0b1b90 0x7fde1c0ad680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e9b0 tx=0x7fde0400ee70 comp rx=0 tx=0).stop 2024-03-08T08:27:25.423 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 shutdown_connections 2024-03-08T08:27:25.423 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fddfc06b440 0x7fddfc06d900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.423 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde1c155280 0x7fde1c0adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.423 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde1c0b1b90 0x7fde1c0ad680 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.423 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 --2- 172.21.15.105:0/2442003367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde1c0b0820 0x7fde1c151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:25.423 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 >> 172.21.15.105:0/2442003367 conn(0x7fde1c0712f0 msgr2=0x7fde1c06b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:25.423 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 shutdown_connections 2024-03-08T08:27:25.423 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:25.421+0000 7fde21b3e640 1 -- 172.21.15.105:0/2442003367 wait complete. 2024-03-08T08:27:25.431 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-08T08:27:32.237 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:32.337 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.336+0000 7f3f0fa78640 1 Processor -- start 2024-03-08T08:27:32.337 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.336+0000 7f3f0fa78640 1 -- start start 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0fa78640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f080cee70 0x7f3f080d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0fa78640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f080cf2d0 con 0x7f3f080d1ee0 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0fa78640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0806bff0 con 0x7f3f080cea90 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0fa78640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0806cc00 con 0x7f3f080cee70 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0e015640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3f080d1ee0 0x7f3f0815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50836/0 (socket says 172.21.15.105:50836) 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0e015640 1 -- 172.21.15.105:0/3930107897 learned_addr learned my addr 172.21.15.105:0/3930107897 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0d013640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f080cee70 0x7f3f080d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270287468 0 0) 0x7f3f080cf2d0 con 0x7f3f080d1ee0 2024-03-08T08:27:32.341 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.337+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef4003520 con 0x7f3f080d1ee0 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334327198 0 0) 0x7f3f0806bff0 con 0x7f3f080cea90 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f080cf2d0 con 0x7f3f080cea90 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2695362655 0 0) 0x7f3ef4003520 con 0x7f3f080d1ee0 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f080cee70 msgr2=0x7f3f080d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f080cee70 0x7f3f080d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 >> v1:172.21.15.144:6789/0 conn(0x7f3f080cea90 legacy=0x7f3f0815ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0806c2d0 con 0x7f3f080d1ee0 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f3f00001670 con 0x7f3f080d1ee0 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 >> v1:172.21.15.105:6789/0 conn(0x7f3f080d1ee0 legacy=0x7f3f0815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef4003b70 0x7f3ef4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ef4004800 0x7f3ef4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f080cee70 0x7f3ef40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0806cc00 con 0x7f3ef4004800 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef400ba70 con 0x7f3ef4003b70 2024-03-08T08:27:32.342 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef400bc40 con 0x7f3f080cee70 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0d013640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ef4004800 0x7f3ef4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0d814640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef4003b70 0x7f3ef4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0e015640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f080cee70 0x7f3ef40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0d013640 1 -- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f080cee70 msgr2=0x7f3ef40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0d013640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f080cee70 0x7f3ef40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0d013640 1 -- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef4003b70 msgr2=0x7f3ef4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0d013640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ef4003b70 0x7f3ef4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.338+0000 7f3f0d013640 1 -- 172.21.15.105:0/3930107897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef4003520 con 0x7f3ef4004800 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.339+0000 7f3f0d013640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ef4004800 0x7f3ef4004c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009ac0 tx=0x7f3ef4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69aa54fab4ba748a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.339+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8039070 con 0x7f3ef4004800 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.339+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef802c660 con 0x7f3ef4004800 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.339+0000 7f3f0c812640 1 -- 172.21.15.105:0/3930107897 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8009690 con 0x7f3ef4004800 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.340+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3930107897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ef4004800 msgr2=0x7f3ef4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.343 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.340+0000 7f3f0fa78640 1 --2- 172.21.15.105:0/3930107897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ef4004800 0x7f3ef4004c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009ac0 tx=0x7f3ef4005830 comp rx=0 tx=0).stop 2024-03-08T08:27:32.344 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.340+0000 7f3f0e015640 1 -- 172.21.15.105:0/3930107897 reap_dead start 2024-03-08T08:27:32.344 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.340+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3930107897 shutdown_connections 2024-03-08T08:27:32.344 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.340+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3930107897 >> 172.21.15.105:0/3930107897 conn(0x7f3f0809aa20 msgr2=0x7f3f0809ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:32.344 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.340+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3930107897 shutdown_connections 2024-03-08T08:27:32.344 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.340+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3930107897 wait complete. 2024-03-08T08:27:32.346 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.341+0000 7f3f0fa78640 1 Processor -- start 2024-03-08T08:27:32.346 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.341+0000 7f3f0fa78640 1 -- start start 2024-03-08T08:27:32.346 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0fa78640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ef4004800 0x7f3f0815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0fa78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f080cb7d0 0x7f3f080c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0fa78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f0815c2d0 0x7f3f0815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0fa78640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0806d090 con 0x7f3f080cb7d0 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0fa78640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0806c890 con 0x7f3f0815c2d0 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0fa78640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080d0b80 con 0x7f3ef4004800 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f080cb7d0 0x7f3f080c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f080cb7d0 0x7f3f080c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51664/0 (socket says 172.21.15.105:51664) 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 -- 172.21.15.105:0/3092053531 learned_addr learned my addr 172.21.15.105:0/3092053531 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d814640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ef4004800 0x7f3f0815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0e015640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f0815c2d0 0x7f3f0815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 -- 172.21.15.105:0/3092053531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ef4004800 msgr2=0x7f3f0815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ef4004800 0x7f3f0815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 -- 172.21.15.105:0/3092053531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f0815c2d0 msgr2=0x7f3f0815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.347 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f0815c2d0 0x7f3f0815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 -- 172.21.15.105:0/3092053531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0815b230 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3f0d013640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f080cb7d0 0x7f3f080c9710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8037040 tx=0x7f3ef8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.342+0000 7f3efe7fc640 1 -- 172.21.15.105:0/3092053531 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8039050 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.343+0000 7f3efe7fc640 1 -- 172.21.15.105:0/3092053531 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef8045070 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.343+0000 7f3efe7fc640 1 -- 172.21.15.105:0/3092053531 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8003830 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.343+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0815b510 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.343+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0815bad0 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.343+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0806a380 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.344+0000 7f3efe7fc640 1 -- 172.21.15.105:0/3092053531 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ef80039d0 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.344+0000 7f3efe7fc640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3eec06b4d0 0x7f3eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.344+0000 7f3efe7fc640 1 -- 172.21.15.105:0/3092053531 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ef80ab800 con 0x7f3f080cb7d0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.344+0000 7f3f0d814640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3eec06b4d0 0x7f3eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.348 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.345+0000 7f3f0d814640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3eec06b4d0 0x7f3eec06d990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f04001fd0 tx=0x7f3f04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.350 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.349+0000 7f3efe7fc640 1 -- 172.21.15.105:0/3092053531 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ef804e020 con 0x7f3f080cb7d0 2024-03-08T08:27:32.526 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.526+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f080cf950 con 0x7f3f080cb7d0 2024-03-08T08:27:32.527 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.526+0000 7f3efe7fc640 1 -- 172.21.15.105:0/3092053531 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12896 (secure 0 0 0) 0x7f3ef807a380 con 0x7f3f080cb7d0 2024-03-08T08:27:32.528 INFO:teuthology.misc.health.smithi105.stdout: 2024-03-08T08:27:32.528 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":16,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:29.420440+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3eec06b4d0 msgr2=0x7f3eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3eec06b4d0 0x7f3eec06d990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f04001fd0 tx=0x7f3f04009040 comp rx=0 tx=0).stop 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f080cb7d0 msgr2=0x7f3f080c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f080cb7d0 0x7f3f080c9710 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8037040 tx=0x7f3ef8004730 comp rx=0 tx=0).stop 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 shutdown_connections 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3eec06b4d0 0x7f3eec06d990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f0815c2d0 0x7f3f0815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f080cb7d0 0x7f3f080c9710 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 --2- 172.21.15.105:0/3092053531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ef4004800 0x7f3f0815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.530 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 >> 172.21.15.105:0/3092053531 conn(0x7f3f0809aa20 msgr2=0x7f3f080b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:32.531 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 shutdown_connections 2024-03-08T08:27:32.531 INFO:teuthology.misc.health.smithi105.stderr:2024-03-08T08:27:32.529+0000 7f3f0fa78640 1 -- 172.21.15.105:0/3092053531 wait complete. 2024-03-08T08:27:32.539 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-08T08:27:32.539 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-08T08:27:32.540 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-08T08:27:32.540 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-08T08:27:32.542 INFO:tasks.ceph.mds.a:Started 2024-03-08T08:27:32.542 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-08T08:27:32.542 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-08T08:27:32.544 INFO:tasks.ceph.mds.c:Started 2024-03-08T08:27:32.544 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-08T08:27:32.544 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-08T08:27:32.546 INFO:tasks.ceph.mds.b:Started 2024-03-08T08:27:32.547 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-08T08:27:32.547 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-08T08:27:32.549 INFO:tasks.ceph.mds.d:Started 2024-03-08T08:27:32.551 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-08T08:27:32.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:32.616 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-03-08T08:27:32.617 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-03-08T08:27:32.633 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-03-08T08:27:32.633 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-03-08T08:27:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.656+0000 7ff27abfd640 1 Processor -- start 2024-03-08T08:27:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.656+0000 7ff27abfd640 1 -- start start 2024-03-08T08:27:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.656+0000 7ff27abfd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff27415df20 0x7ff27406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.656+0000 7ff27919a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff27406b390 0x7ff2740ad970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50868/0 (socket says 172.21.15.105:50868) 2024-03-08T08:27:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.656+0000 7ff27919a640 1 -- 172.21.15.105:0/2286696462 learned_addr learned my addr 172.21.15.105:0/2286696462 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.656+0000 7ff27abfd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27406f830 con 0x7ff27406b390 2024-03-08T08:27:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff27abfd640 1 -- 172.21.15.105:0/2286696462 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27406fad0 con 0x7ff2740b5990 2024-03-08T08:27:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff27abfd640 1 -- 172.21.15.105:0/2286696462 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27406dbb0 con 0x7ff27415df20 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff278999640 1 --2- 172.21.15.105:0/2286696462 >> v2:172.21.15.144:3300/0 conn(0x7ff27415df20 0x7ff27406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff2737fe640 1 -- 172.21.15.105:0/2286696462 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133402934 0 0) 0x7ff27406f830 con 0x7ff27406b390 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff2737fe640 1 -- 172.21.15.105:0/2286696462 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff264003520 con 0x7ff27406b390 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff2737fe640 1 -- 172.21.15.105:0/2286696462 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937618668 0 0) 0x7ff27406fad0 con 0x7ff2740b5990 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff2737fe640 1 -- 172.21.15.105:0/2286696462 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff27406f830 con 0x7ff2740b5990 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff2737fe640 1 -- 172.21.15.105:0/2286696462 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100943105 0 0) 0x7ff264003520 con 0x7ff27406b390 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff278999640 1 -- 172.21.15.105:0/2286696462 >> v1:172.21.15.144:6789/0 conn(0x7ff2740b5990 legacy=0x7ff27406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff278999640 1 -- 172.21.15.105:0/2286696462 >> v1:172.21.15.105:6789/0 conn(0x7ff27406b390 legacy=0x7ff2740ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.657+0000 7ff278999640 1 -- 172.21.15.105:0/2286696462 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27406fdb0 con 0x7ff27415df20 2024-03-08T08:27:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff278999640 1 --2- 172.21.15.105:0/2286696462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 0x7ff27406c8b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2680097a0 tx=0x7ff26802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9caf8a891ea4d483 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff2737fe640 1 -- 172.21.15.105:0/2286696462 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268037070 con 0x7ff27415df20 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff2737fe640 1 -- 172.21.15.105:0/2286696462 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268030510 con 0x7ff27415df20 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff2737fe640 1 -- 172.21.15.105:0/2286696462 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c001a10 con 0x7ff27415df20 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff27abfd640 1 -- 172.21.15.105:0/2286696462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 msgr2=0x7ff27406c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff27abfd640 1 --2- 172.21.15.105:0/2286696462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 0x7ff27406c8b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2680097a0 tx=0x7ff26802b790 comp rx=0 tx=0).stop 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff27abfd640 1 -- 172.21.15.105:0/2286696462 shutdown_connections 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff27abfd640 1 --2- 172.21.15.105:0/2286696462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 0x7ff27406c8b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff27abfd640 1 -- 172.21.15.105:0/2286696462 >> 172.21.15.105:0/2286696462 conn(0x7ff27406bd90 msgr2=0x7ff27406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff27abfd640 1 -- 172.21.15.105:0/2286696462 shutdown_connections 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.658+0000 7ff27abfd640 1 -- 172.21.15.105:0/2286696462 wait complete. 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff27abfd640 1 Processor -- start 2024-03-08T08:27:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff27abfd640 1 -- start start 2024-03-08T08:27:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff27abfd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff27406b390 0x7ff274155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff27abfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2740b5990 0x7ff274146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff27abfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 0x7ff2741573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff27abfd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740ae080 con 0x7ff2740b5990 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff27abfd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27406e0e0 con 0x7ff27415df20 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff27abfd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740ae200 con 0x7ff27406b390 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.660+0000 7ff273fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2740b5990 0x7ff274146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff273fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2740b5990 0x7ff274146420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51710/0 (socket says 172.21.15.105:51710) 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff273fff640 1 -- 172.21.15.105:0/3002441186 learned_addr learned my addr 172.21.15.105:0/3002441186 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff278999640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff27406b390 0x7ff274155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff27919a640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 0x7ff2741573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff273fff640 1 -- 172.21.15.105:0/3002441186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff27406b390 msgr2=0x7ff274155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff273fff640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff27406b390 0x7ff274155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff273fff640 1 -- 172.21.15.105:0/3002441186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 msgr2=0x7ff2741573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff273fff640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 0x7ff2741573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff273fff640 1 -- 172.21.15.105:0/3002441186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274146960 con 0x7ff2740b5990 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff273fff640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2740b5990 0x7ff274146420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff26000c490 tx=0x7ff26000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff2717fa640 1 -- 172.21.15.105:0/3002441186 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2600052d0 con 0x7ff2740b5990 2024-03-08T08:27:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff2717fa640 1 -- 172.21.15.105:0/3002441186 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff260005470 con 0x7ff2740b5990 2024-03-08T08:27:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff2717fa640 1 -- 172.21.15.105:0/3002441186 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260012850 con 0x7ff2740b5990 2024-03-08T08:27:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27415e4a0 con 0x7ff2740b5990 2024-03-08T08:27:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.661+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27415ea60 con 0x7ff2740b5990 2024-03-08T08:27:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.662+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff274067df0 con 0x7ff2740b5990 2024-03-08T08:27:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.663+0000 7ff2717fa640 1 -- 172.21.15.105:0/3002441186 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2600129f0 con 0x7ff2740b5990 2024-03-08T08:27:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.663+0000 7ff2717fa640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff25806b4d0 0x7ff25806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.663+0000 7ff2717fa640 1 -- 172.21.15.105:0/3002441186 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff260089990 con 0x7ff2740b5990 2024-03-08T08:27:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.663+0000 7ff278999640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff25806b4d0 0x7ff25806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.664+0000 7ff278999640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff25806b4d0 0x7ff25806d990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff268034000 tx=0x7ff26802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.667+0000 7ff2717fa640 1 -- 172.21.15.105:0/3002441186 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff274067df0 con 0x7ff2740b5990 2024-03-08T08:27:32.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.839+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff274075120 con 0x7ff2740b5990 2024-03-08T08:27:32.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.839+0000 7ff2717fa640 1 -- 172.21.15.105:0/3002441186 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12896 (secure 0 0 0) 0x7ff2600585a0 con 0x7ff2740b5990 2024-03-08T08:27:32.840 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:32.841 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:29.420440+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:32.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff25806b4d0 msgr2=0x7ff25806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff25806b4d0 0x7ff25806d990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff268034000 tx=0x7ff26802f000 comp rx=0 tx=0).stop 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2740b5990 msgr2=0x7ff274146420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2740b5990 0x7ff274146420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff26000c490 tx=0x7ff26000c950 comp rx=0 tx=0).stop 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 shutdown_connections 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff25806b4d0 0x7ff25806d990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff27415df20 0x7ff2741573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2740b5990 0x7ff274146420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 --2- 172.21.15.105:0/3002441186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff27406b390 0x7ff274155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 >> 172.21.15.105:0/3002441186 conn(0x7ff27406bd90 msgr2=0x7ff2740ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 shutdown_connections 2024-03-08T08:27:32.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.841+0000 7ff27abfd640 1 -- 172.21.15.105:0/3002441186 wait complete. 2024-03-08T08:27:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:27:32.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:27:32.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.955+0000 7fc44de9d640 1 Processor -- start 2024-03-08T08:27:32.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.955+0000 7fc44de9d640 1 -- start start 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc44de9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc4480d43e0 0x7fc4480b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc44de9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc4480d2910 0x7fc44815de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc44de9d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4480d48f0 con 0x7fc4480d2530 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc44de9d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4480d4bb0 con 0x7fc4480d43e0 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc44de9d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4480ceb30 con 0x7fc4480d2910 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc446ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc4480d43e0 0x7fc4480b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc446ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc4480d43e0 0x7fc4480b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51724/0 (socket says 172.21.15.105:51724) 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc446ffd640 1 -- 172.21.15.105:0/2334629642 learned_addr learned my addr 172.21.15.105:0/2334629642 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc447fff640 1 --2- 172.21.15.105:0/2334629642 >> v2:172.21.15.144:3300/0 conn(0x7fc4480d2910 0x7fc44815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc446ffd640 1 -- 172.21.15.105:0/2334629642 >> v2:172.21.15.144:3300/0 conn(0x7fc4480d2910 msgr2=0x7fc44815de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc446ffd640 1 --2- 172.21.15.105:0/2334629642 >> v2:172.21.15.144:3300/0 conn(0x7fc4480d2910 0x7fc44815de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc446ffd640 1 -- 172.21.15.105:0/2334629642 >> v1:172.21.15.144:6789/0 conn(0x7fc4480d2530 legacy=0x7fc4480b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc446ffd640 1 -- 172.21.15.105:0/2334629642 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480cee10 con 0x7fc4480d43e0 2024-03-08T08:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.956+0000 7fc447fff640 1 --2- 172.21.15.105:0/2334629642 >> v2:172.21.15.144:3300/0 conn(0x7fc4480d2910 0x7fc44815de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.957+0000 7fc446ffd640 1 --2- 172.21.15.105:0/2334629642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d43e0 0x7fc4480b5170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc448161ee0 tx=0x7fc43c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43b9cb3da6ed238 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.957+0000 7fc4467fc640 1 -- 172.21.15.105:0/2334629642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c035020 con 0x7fc4480d43e0 2024-03-08T08:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.957+0000 7fc4467fc640 1 -- 172.21.15.105:0/2334629642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c031870 con 0x7fc4480d43e0 2024-03-08T08:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.957+0000 7fc4467fc640 1 -- 172.21.15.105:0/2334629642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c031c40 con 0x7fc4480d43e0 2024-03-08T08:27:32.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.957+0000 7fc44de9d640 1 -- 172.21.15.105:0/2334629642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d43e0 msgr2=0x7fc4480b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.957+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2334629642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d43e0 0x7fc4480b5170 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc448161ee0 tx=0x7fc43c02b910 comp rx=0 tx=0).stop 2024-03-08T08:27:32.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.958+0000 7fc44de9d640 1 -- 172.21.15.105:0/2334629642 shutdown_connections 2024-03-08T08:27:32.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.958+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2334629642 >> v2:172.21.15.144:3300/0 conn(0x7fc4480d2910 0x7fc44815de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.958+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2334629642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d43e0 0x7fc4480b5170 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.958+0000 7fc44de9d640 1 -- 172.21.15.105:0/2334629642 >> 172.21.15.105:0/2334629642 conn(0x7fc44809cfb0 msgr2=0x7fc4480d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.958+0000 7fc44de9d640 1 -- 172.21.15.105:0/2334629642 shutdown_connections 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.958+0000 7fc44de9d640 1 -- 172.21.15.105:0/2334629642 wait complete. 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.959+0000 7fc44de9d640 1 Processor -- start 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.959+0000 7fc44de9d640 1 -- start start 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.959+0000 7fc44de9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4480d2530 0x7fc4481605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc44de9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d2910 0x7fc44814ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc44de9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4480d43e0 0x7fc448150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc44de9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44815e340 con 0x7fc4480d2910 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc44de9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480b3080 con 0x7fc4480d43e0 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc44de9d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44815e4c0 con 0x7fc4480d2530 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d2910 0x7fc44814ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d2910 0x7fc44814ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51726/0 (socket says 172.21.15.105:51726) 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 -- 172.21.15.105:0/2825773117 learned_addr learned my addr 172.21.15.105:0/2825773117 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc4477fe640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4480d2530 0x7fc4481605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 -- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4480d2530 msgr2=0x7fc4481605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc447fff640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4480d43e0 0x7fc448150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4480d2530 0x7fc4481605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 -- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4480d43e0 msgr2=0x7fc448150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4480d43e0 0x7fc448150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 -- 172.21.15.105:0/2825773117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc448161050 con 0x7fc4480d2910 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc4477fe640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4480d2530 0x7fc4481605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc446ffd640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d2910 0x7fc44814ff00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc43c02b950 tx=0x7fc43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc42ffff640 1 -- 172.21.15.105:0/2825773117 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c035040 con 0x7fc4480d2910 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc42ffff640 1 -- 172.21.15.105:0/2825773117 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c045480 con 0x7fc4480d2910 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc448161330 con 0x7fc4480d2910 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc42ffff640 1 -- 172.21.15.105:0/2825773117 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c045870 con 0x7fc4480d2910 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.960+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4481618f0 con 0x7fc4480d2910 2024-03-08T08:27:32.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.961+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc448075120 con 0x7fc4480d2910 2024-03-08T08:27:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.962+0000 7fc42ffff640 1 -- 172.21.15.105:0/2825773117 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc43c038020 con 0x7fc4480d2910 2024-03-08T08:27:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.962+0000 7fc42ffff640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc43406b4d0 0x7fc43406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.962+0000 7fc42ffff640 1 -- 172.21.15.105:0/2825773117 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc43c0ac2c0 con 0x7fc4480d2910 2024-03-08T08:27:32.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.962+0000 7fc4477fe640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc43406b4d0 0x7fc43406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.962+0000 7fc4477fe640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc43406b4d0 0x7fc43406d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc438001680 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:32.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:32.967+0000 7fc42ffff640 1 -- 172.21.15.105:0/2825773117 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc43c07b520 con 0x7fc4480d2910 2024-03-08T08:27:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.137+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4480ce220 con 0x7fc4480d2910 2024-03-08T08:27:33.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.137+0000 7fc42ffff640 1 -- 172.21.15.105:0/2825773117 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc43c07ae20 con 0x7fc4480d2910 2024-03-08T08:27:33.138 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc43406b4d0 msgr2=0x7fc43406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc43406b4d0 0x7fc43406d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc438001680 tx=0x7fc438009040 comp rx=0 tx=0).stop 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d2910 msgr2=0x7fc44814ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d2910 0x7fc44814ff00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc43c02b950 tx=0x7fc43c0046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 shutdown_connections 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc43406b4d0 0x7fc43406d990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4480d43e0 0x7fc448150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4480d2910 0x7fc44814ff00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 --2- 172.21.15.105:0/2825773117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4480d2530 0x7fc4481605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 >> 172.21.15.105:0/2825773117 conn(0x7fc44809cfb0 msgr2=0x7fc4480b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 shutdown_connections 2024-03-08T08:27:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.139+0000 7fc44de9d640 1 -- 172.21.15.105:0/2825773117 wait complete. 2024-03-08T08:27:33.151 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:27:33.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.250+0000 7fec31aae640 1 Processor -- start 2024-03-08T08:27:33.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.250+0000 7fec31aae640 1 -- start start 2024-03-08T08:27:33.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec31aae640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec2c0b1550 0x7fec2c0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:33.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec31aae640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c0b1af0 con 0x7fec2c15dd80 2024-03-08T08:27:33.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec31aae640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c0b1dd0 con 0x7fec2c0b2b90 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec31aae640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c0b2700 con 0x7fec2c0b1550 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec2b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fec2c15dd80 0x7fec2c1623c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50872/0 (socket says 172.21.15.105:50872) 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec2b7fe640 1 -- 172.21.15.105:0/1405770102 learned_addr learned my addr 172.21.15.105:0/1405770102 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148864305 0 0) 0x7fec2c0b1af0 con 0x7fec2c15dd80 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec14003520 con 0x7fec2c15dd80 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.251+0000 7fec2affd640 1 --2- 172.21.15.105:0/1405770102 >> v2:172.21.15.144:3300/0 conn(0x7fec2c0b1550 0x7fec2c0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1978032628 0 0) 0x7fec14003520 con 0x7fec2c15dd80 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 >> v2:172.21.15.144:3300/0 conn(0x7fec2c0b1550 msgr2=0x7fec2c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 --2- 172.21.15.105:0/1405770102 >> v2:172.21.15.144:3300/0 conn(0x7fec2c0b1550 0x7fec2c0aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 >> v1:172.21.15.144:6789/0 conn(0x7fec2c0b2b90 legacy=0x7fec2c0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c0b20b0 con 0x7fec2c15dd80 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fec1c001670 con 0x7fec2c15dd80 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2affd640 1 --2- 172.21.15.105:0/1405770102 >> v2:172.21.15.144:3300/0 conn(0x7fec2c0b1550 0x7fec2c0aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 >> v1:172.21.15.105:6789/0 conn(0x7fec2c15dd80 legacy=0x7fec2c1623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec140039e0 0x7fec14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec14004800 0x7fec14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0b2700 con 0x7fec14005210 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1400c140 con 0x7fec140039e0 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1400c340 con 0x7fec14004800 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2bfff640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2b7fe640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec140039e0 0x7fec14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2affd640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec14004800 0x7fec14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2bfff640 1 -- 172.21.15.105:0/1405770102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec14004800 msgr2=0x7fec14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2bfff640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec14004800 0x7fec14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2bfff640 1 -- 172.21.15.105:0/1405770102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec140039e0 msgr2=0x7fec14003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2bfff640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec140039e0 0x7fec14003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.252+0000 7fec2bfff640 1 -- 172.21.15.105:0/1405770102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec14003520 con 0x7fec14005210 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec2bfff640 1 -- 172.21.15.105:0/1405770102 reap_dead start 2024-03-08T08:27:33.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec2bfff640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec1400b9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec24001e20 tx=0x7fec14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f30af0c4da756f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24001a70 con 0x7fec14005210 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24003b70 con 0x7fec14005210 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec2a7fc640 1 -- 172.21.15.105:0/1405770102 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240050d0 con 0x7fec14005210 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec31aae640 1 -- 172.21.15.105:0/1405770102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 msgr2=0x7fec1400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec31aae640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec1400b9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec24001e20 tx=0x7fec14005830 comp rx=0 tx=0).stop 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec31aae640 1 -- 172.21.15.105:0/1405770102 shutdown_connections 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec31aae640 1 --2- 172.21.15.105:0/1405770102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec1400b9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec31aae640 1 -- 172.21.15.105:0/1405770102 >> 172.21.15.105:0/1405770102 conn(0x7fec2c0b37a0 msgr2=0x7fec2c0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec31aae640 1 -- 172.21.15.105:0/1405770102 shutdown_connections 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.253+0000 7fec31aae640 1 -- 172.21.15.105:0/1405770102 wait complete. 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.254+0000 7fec31aae640 1 Processor -- start 2024-03-08T08:27:33.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec31aae640 1 -- start start 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec31aae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec2c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec31aae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0cae20 0x7fec2c0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec31aae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0cdd90 0x7fec2c0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec31aae640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c15e160 con 0x7fec14005210 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec31aae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c1623c0 con 0x7fec2c0cdd90 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec31aae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c1f79c0 con 0x7fec2c0cae20 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec2b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec2c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec2b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec2c155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51744/0 (socket says 172.21.15.105:51744) 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.255+0000 7fec2b7fe640 1 -- 172.21.15.105:0/639523860 learned_addr learned my addr 172.21.15.105:0/639523860 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec2affd640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0cae20 0x7fec2c0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec2bfff640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0cdd90 0x7fec2c0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:33.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec2b7fe640 1 -- 172.21.15.105:0/639523860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0cae20 msgr2=0x7fec2c0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec2b7fe640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0cae20 0x7fec2c0cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec2b7fe640 1 -- 172.21.15.105:0/639523860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0cdd90 msgr2=0x7fec2c0ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec2b7fe640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0cdd90 0x7fec2c0ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec2b7fe640 1 -- 172.21.15.105:0/639523860 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c1f7ca0 con 0x7fec14005210 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec2b7fe640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec2c155620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec1c007290 tx=0x7fec1c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec13fff640 1 -- 172.21.15.105:0/639523860 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00f960 con 0x7fec14005210 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec13fff640 1 -- 172.21.15.105:0/639523860 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c018070 con 0x7fec14005210 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c1f7f80 con 0x7fec14005210 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec13fff640 1 -- 172.21.15.105:0/639523860 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00fd40 con 0x7fec14005210 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.256+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c0ce750 con 0x7fec14005210 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.257+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c0cb7e0 con 0x7fec14005210 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.257+0000 7fec13fff640 1 -- 172.21.15.105:0/639523860 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec1c01d460 con 0x7fec14005210 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.258+0000 7fec13fff640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec1806b4d0 0x7fec1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:33.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.258+0000 7fec2affd640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec1806b4d0 0x7fec1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:33.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.258+0000 7fec13fff640 1 -- 172.21.15.105:0/639523860 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec1c089e30 con 0x7fec14005210 2024-03-08T08:27:33.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.258+0000 7fec2affd640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec1806b4d0 0x7fec1806d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec2c1584b0 tx=0x7fec20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:33.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.262+0000 7fec13fff640 1 -- 172.21.15.105:0/639523860 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec1c059000 con 0x7fec14005210 2024-03-08T08:27:33.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.470+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec2c075120 con 0x7fec14005210 2024-03-08T08:27:33.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.470+0000 7fec13fff640 1 -- 172.21.15.105:0/639523860 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7fec1c058900 con 0x7fec14005210 2024-03-08T08:27:33.472 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:33.473 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T08:27:33.473 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 2 2024-03-08T08:27:33.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec1806b4d0 msgr2=0x7fec1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec1806b4d0 0x7fec1806d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec2c1584b0 tx=0x7fec20008040 comp rx=0 tx=0).stop 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 msgr2=0x7fec2c155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec2c155620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec1c007290 tx=0x7fec1c007750 comp rx=0 tx=0).stop 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 shutdown_connections 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec1806b4d0 0x7fec1806d990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0cdd90 0x7fec2c0ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c0cae20 0x7fec2c0cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 --2- 172.21.15.105:0/639523860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec14005210 0x7fec2c155620 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 >> 172.21.15.105:0/639523860 conn(0x7fec2c0b37a0 msgr2=0x7fec2c06f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 shutdown_connections 2024-03-08T08:27:33.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:33.472+0000 7fec31aae640 1 -- 172.21.15.105:0/639523860 wait complete. 2024-03-08T08:27:35.486 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:27:35.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.595+0000 7f9428610640 1 Processor -- start 2024-03-08T08:27:35.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.595+0000 7f9428610640 1 -- start start 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9428610640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9428610640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94200d43e0 0x7f94200b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9428610640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f94200d2910 0x7f942015de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9428610640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200b3880 con 0x7f94200d2910 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9428610640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94200d48f0 con 0x7f94200d43e0 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9428610640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94200d4bb0 con 0x7f94200d2530 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9426bad640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f94200d2910 0x7f942015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9426bad640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f94200d2910 0x7f942015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58670/0 (socket says 172.21.15.105:58670) 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9426bad640 1 -- 172.21.15.105:0/3144137883 learned_addr learned my addr 172.21.15.105:0/3144137883 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f94263ac640 1 --2- 172.21.15.105:0/3144137883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.596+0000 7f9425bab640 1 --2- 172.21.15.105:0/3144137883 >> v2:172.21.15.144:3300/0 conn(0x7f94200d43e0 0x7f94200b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.597+0000 7f9426bad640 1 -- 172.21.15.105:0/3144137883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 msgr2=0x7f94200b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:35.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.597+0000 7f9426bad640 1 --2- 172.21.15.105:0/3144137883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.597+0000 7f9426bad640 1 -- 172.21.15.105:0/3144137883 >> v2:172.21.15.144:3300/0 conn(0x7f94200d43e0 msgr2=0x7f94200b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:35.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.597+0000 7f9426bad640 1 --2- 172.21.15.105:0/3144137883 >> v2:172.21.15.144:3300/0 conn(0x7f94200d43e0 0x7f94200b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.597+0000 7f9426bad640 1 -- 172.21.15.105:0/3144137883 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94200ceb30 con 0x7f94200d2910 2024-03-08T08:27:35.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.597+0000 7f9426bad640 1 --2- 172.21.15.105:0/3144137883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f942015de00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f942008c380 tx=0x7f941802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce3919fed4eb030 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:35.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f94253aa640 1 -- 172.21.15.105:0/3144137883 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418038070 con 0x7f94200d2910 2024-03-08T08:27:35.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f94253aa640 1 -- 172.21.15.105:0/3144137883 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941802f050 con 0x7f94200d2910 2024-03-08T08:27:35.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f94253aa640 1 -- 172.21.15.105:0/3144137883 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418031970 con 0x7f94200d2910 2024-03-08T08:27:35.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 -- 172.21.15.105:0/3144137883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 msgr2=0x7f942015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 --2- 172.21.15.105:0/3144137883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f942015de00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f942008c380 tx=0x7f941802b7f0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 -- 172.21.15.105:0/3144137883 shutdown_connections 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 --2- 172.21.15.105:0/3144137883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f942015de00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 --2- 172.21.15.105:0/3144137883 >> v2:172.21.15.144:3300/0 conn(0x7f94200d43e0 0x7f94200b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 --2- 172.21.15.105:0/3144137883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 -- 172.21.15.105:0/3144137883 >> 172.21.15.105:0/3144137883 conn(0x7f942009cfb0 msgr2=0x7f94200d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 -- 172.21.15.105:0/3144137883 shutdown_connections 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.598+0000 7f9428610640 1 -- 172.21.15.105:0/3144137883 wait complete. 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.599+0000 7f9428610640 1 Processor -- start 2024-03-08T08:27:35.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9428610640 1 -- start start 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9428610640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9428610640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f9420161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9428610640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94200d43e0 0x7f94200b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9428610640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94200b3080 con 0x7f94200d2910 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9428610640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94200cf0f0 con 0x7f94200d43e0 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9428610640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942015e340 con 0x7f94200d2530 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9425bab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f9420161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9425bab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f9420161050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58684/0 (socket says 172.21.15.105:58684) 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f9425bab640 1 -- 172.21.15.105:0/1236197435 learned_addr learned my addr 172.21.15.105:0/1236197435 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.600+0000 7f94263ac640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9426bad640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94200d43e0 0x7f94200b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9425bab640 1 -- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 msgr2=0x7f94200b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:35.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9425bab640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9425bab640 1 -- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94200d43e0 msgr2=0x7f94200b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9425bab640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94200d43e0 0x7f94200b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9425bab640 1 -- 172.21.15.105:0/1236197435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94200b6e90 con 0x7f94200d2910 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f94263ac640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9425bab640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f9420161050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f941000d5f0 tx=0x7f941000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9416ffd640 1 -- 172.21.15.105:0/1236197435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94100125f0 con 0x7f94200d2910 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9416ffd640 1 -- 172.21.15.105:0/1236197435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410019070 con 0x7f94200d2910 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94200b71d0 con 0x7f94200d2910 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9416ffd640 1 -- 172.21.15.105:0/1236197435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94100129d0 con 0x7f94200d2910 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.601+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94200b7a10 con 0x7f94200d2910 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.602+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420075120 con 0x7f94200d2910 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.602+0000 7f9416ffd640 1 -- 172.21.15.105:0/1236197435 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9410012b70 con 0x7f94200d2910 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.603+0000 7f9416ffd640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f940406b4d0 0x7f940406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.603+0000 7f94263ac640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f940406b4d0 0x7f940406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.603+0000 7f9416ffd640 1 -- 172.21.15.105:0/1236197435 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f941008a300 con 0x7f94200d2910 2024-03-08T08:27:35.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.603+0000 7f94263ac640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f940406b4d0 0x7f940406d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f941c009aa0 tx=0x7f941c0039f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:35.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.607+0000 7f9416ffd640 1 -- 172.21.15.105:0/1236197435 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94200b71d0 con 0x7f94200d2910 2024-03-08T08:27:35.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.812+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94200ce220 con 0x7f94200d2910 2024-03-08T08:27:35.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.813+0000 7f9416ffd640 1 -- 172.21.15.105:0/1236197435 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3706 (secure 0 0 0) 0x7f9410059440 con 0x7f94200d2910 2024-03-08T08:27:35.815 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:35.815 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-08T08:27:35.815 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 3 2024-03-08T08:27:35.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f940406b4d0 msgr2=0x7f940406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f940406b4d0 0x7f940406d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f941c009aa0 tx=0x7f941c0039f0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 msgr2=0x7f9420161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f9420161050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f941000d5f0 tx=0x7f941000dab0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 shutdown_connections 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f940406b4d0 0x7f940406d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94200d43e0 0x7f94200b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94200d2910 0x7f9420161050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 --2- 172.21.15.105:0/1236197435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94200d2530 0x7f94200b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 >> 172.21.15.105:0/1236197435 conn(0x7f942009cfb0 msgr2=0x7f94200d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 shutdown_connections 2024-03-08T08:27:35.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.814+0000 7f9428610640 1 -- 172.21.15.105:0/1236197435 wait complete. 2024-03-08T08:27:35.827 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.924+0000 7fe243aae640 1 Processor -- start 2024-03-08T08:27:35.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.924+0000 7fe243aae640 1 -- start start 2024-03-08T08:27:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe243aae640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe23c1615d0 0x7fe23c0b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe243aae640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe23c0d05a0 0x7fe23c0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe243aae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c0ce8e0 0x7fe23c0ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe243aae640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c0d0f40 con 0x7fe23c0d05a0 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe243aae640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c0cf660 con 0x7fe23c1615d0 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe243aae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0cf940 con 0x7fe23c0ce8e0 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe241049640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe23c0d05a0 0x7fe23c0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe241049640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe23c0d05a0 0x7fe23c0d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58698/0 (socket says 172.21.15.105:58698) 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.925+0000 7fe241049640 1 -- 172.21.15.105:0/988682687 learned_addr learned my addr 172.21.15.105:0/988682687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe24204b640 1 --2- 172.21.15.105:0/988682687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c0ce8e0 0x7fe23c0ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe24184a640 1 --2- 172.21.15.105:0/988682687 >> v2:172.21.15.144:3300/0 conn(0x7fe23c1615d0 0x7fe23c0b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe241049640 1 -- 172.21.15.105:0/988682687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c0ce8e0 msgr2=0x7fe23c0ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe241049640 1 --2- 172.21.15.105:0/988682687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c0ce8e0 0x7fe23c0ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe241049640 1 -- 172.21.15.105:0/988682687 >> v2:172.21.15.144:3300/0 conn(0x7fe23c1615d0 msgr2=0x7fe23c0b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe241049640 1 --2- 172.21.15.105:0/988682687 >> v2:172.21.15.144:3300/0 conn(0x7fe23c1615d0 0x7fe23c0b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe241049640 1 -- 172.21.15.105:0/988682687 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0cfc20 con 0x7fe23c0d05a0 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe24184a640 1 --2- 172.21.15.105:0/988682687 >> v2:172.21.15.144:3300/0 conn(0x7fe23c1615d0 0x7fe23c0b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe241049640 1 --2- 172.21.15.105:0/988682687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0d0a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe23c08c780 tx=0x7fe22c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7877852f7bb5920b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:35.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.926+0000 7fe240848640 1 -- 172.21.15.105:0/988682687 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c035020 con 0x7fe23c0d05a0 2024-03-08T08:27:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe240848640 1 -- 172.21.15.105:0/988682687 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c031870 con 0x7fe23c0d05a0 2024-03-08T08:27:35.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe240848640 1 -- 172.21.15.105:0/988682687 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c031c40 con 0x7fe23c0d05a0 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 -- 172.21.15.105:0/988682687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 msgr2=0x7fe23c0d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 --2- 172.21.15.105:0/988682687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0d0a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe23c08c780 tx=0x7fe22c02b910 comp rx=0 tx=0).stop 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 -- 172.21.15.105:0/988682687 shutdown_connections 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 --2- 172.21.15.105:0/988682687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c0ce8e0 0x7fe23c0ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 --2- 172.21.15.105:0/988682687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0d0a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 --2- 172.21.15.105:0/988682687 >> v2:172.21.15.144:3300/0 conn(0x7fe23c1615d0 0x7fe23c0b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 -- 172.21.15.105:0/988682687 >> 172.21.15.105:0/988682687 conn(0x7fe23c09cfa0 msgr2=0x7fe23c0d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 -- 172.21.15.105:0/988682687 shutdown_connections 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.927+0000 7fe243aae640 1 -- 172.21.15.105:0/988682687 wait complete. 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.928+0000 7fe243aae640 1 Processor -- start 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.928+0000 7fe243aae640 1 -- start start 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe243aae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe23c0ce8e0 0x7fe23c14b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe243aae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe243aae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c1615d0 0x7fe23c1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe243aae640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0b7060 con 0x7fe23c0d05a0 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe243aae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0d1c20 con 0x7fe23c0ce8e0 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe243aae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c1619b0 con 0x7fe23c1615d0 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58712/0 (socket says 172.21.15.105:58712) 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 -- 172.21.15.105:0/4278797232 learned_addr learned my addr 172.21.15.105:0/4278797232 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe24204b640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c1615d0 0x7fe23c1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe24184a640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe23c0ce8e0 0x7fe23c14b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 -- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c1615d0 msgr2=0x7fe23c1582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c1615d0 0x7fe23c1582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 -- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe23c0ce8e0 msgr2=0x7fe23c14b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe23c0ce8e0 0x7fe23c14b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:35.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 -- 172.21.15.105:0/4278797232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0cb5f0 con 0x7fe23c0d05a0 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe24204b640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c1615d0 0x7fe23c1582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe24184a640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe23c0ce8e0 0x7fe23c14b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.929+0000 7fe241049640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0cada0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe22c000c00 tx=0x7fe22c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.930+0000 7fe2327fc640 1 -- 172.21.15.105:0/4278797232 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c035040 con 0x7fe23c0d05a0 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.930+0000 7fe2327fc640 1 -- 172.21.15.105:0/4278797232 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c030d00 con 0x7fe23c0d05a0 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.930+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c0cb8d0 con 0x7fe23c0d05a0 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.930+0000 7fe2327fc640 1 -- 172.21.15.105:0/4278797232 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0456a0 con 0x7fe23c0d05a0 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.930+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c0cb2e0 con 0x7fe23c0d05a0 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.931+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c0c93b0 con 0x7fe23c0d05a0 2024-03-08T08:27:35.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.931+0000 7fe2327fc640 1 -- 172.21.15.105:0/4278797232 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe22c038020 con 0x7fe23c0d05a0 2024-03-08T08:27:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.931+0000 7fe2327fc640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22006b4d0 0x7fe22006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.931+0000 7fe2327fc640 1 -- 172.21.15.105:0/4278797232 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe22c0ac1e0 con 0x7fe23c0d05a0 2024-03-08T08:27:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.931+0000 7fe24184a640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22006b4d0 0x7fe22006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:35.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.932+0000 7fe24184a640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22006b4d0 0x7fe22006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe238007f40 tx=0x7fe238007490 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:35.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:35.936+0000 7fe2327fc640 1 -- 172.21.15.105:0/4278797232 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe22c02f320 con 0x7fe23c0d05a0 2024-03-08T08:27:36.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.106+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe23c075120 con 0x7fe23c0d05a0 2024-03-08T08:27:36.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.106+0000 7fe2327fc640 1 -- 172.21.15.105:0/4278797232 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12896 (secure 0 0 0) 0x7fe22c07b440 con 0x7fe23c0d05a0 2024-03-08T08:27:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:36.109 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:29.420440+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22006b4d0 msgr2=0x7fe22006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22006b4d0 0x7fe22006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe238007f40 tx=0x7fe238007490 comp rx=0 tx=0).stop 2024-03-08T08:27:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 msgr2=0x7fe23c0cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0cada0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe22c000c00 tx=0x7fe22c0046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 shutdown_connections 2024-03-08T08:27:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22006b4d0 0x7fe22006d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23c1615d0 0x7fe23c1582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe23c0d05a0 0x7fe23c0cada0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 --2- 172.21.15.105:0/4278797232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe23c0ce8e0 0x7fe23c14b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 >> 172.21.15.105:0/4278797232 conn(0x7fe23c09cfa0 msgr2=0x7fe23c15dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:36.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 shutdown_connections 2024-03-08T08:27:36.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.109+0000 7fe243aae640 1 -- 172.21.15.105:0/4278797232 wait complete. 2024-03-08T08:27:36.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:27:36.122 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:27:36.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.222+0000 7fdafac17640 1 Processor -- start 2024-03-08T08:27:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.223+0000 7fdafac17640 1 -- start start 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.223+0000 7fdafac17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf406bcb0 0x7fdaf406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.223+0000 7fdafac17640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40d48a0 con 0x7fdaf4161850 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.223+0000 7fdafac17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40d4b80 con 0x7fdaf4161470 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.223+0000 7fdafac17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf406c170 con 0x7fdaf406bcb0 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.223+0000 7fdaf91b4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdaf4161850 0x7fdaf40b2450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57880/0 (socket says 172.21.15.105:57880) 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.223+0000 7fdaf91b4640 1 -- 172.21.15.105:0/433194919 learned_addr learned my addr 172.21.15.105:0/433194919 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf3fff640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf406bcb0 0x7fdaf406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747764582 0 0) 0x7fdaf40d48a0 con 0x7fdaf4161850 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae4003520 con 0x7fdaf4161850 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699635319 0 0) 0x7fdae4003520 con 0x7fdaf4161850 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf406bcb0 msgr2=0x7fdaf406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf406bcb0 0x7fdaf406c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 >> v1:172.21.15.144:6789/0 conn(0x7fdaf4161470 legacy=0x7fdaf406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf406c450 con 0x7fdaf4161850 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fdaec0019f0 con 0x7fdaf4161850 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 >> v1:172.21.15.105:6789/0 conn(0x7fdaf4161850 legacy=0x7fdaf40b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf406bcb0 0x7fdae40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4003fb0 0x7fdae4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf406c170 con 0x7fdae4004d40 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae400bbb0 con 0x7fdae4003fb0 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae400bdb0 con 0x7fdaf406bcb0 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf91b4640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf89b3640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf406bcb0 0x7fdae40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf3fff640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4003fb0 0x7fdae4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf91b4640 1 -- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf406bcb0 msgr2=0x7fdae40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf91b4640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf406bcb0 0x7fdae40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf91b4640 1 -- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4003fb0 msgr2=0x7fdae4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf91b4640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdae4003fb0 0x7fdae4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.224+0000 7fdaf91b4640 1 -- 172.21.15.105:0/433194919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae4003520 con 0x7fdae4004d40 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.225+0000 7fdaf91b4640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdae40051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00a790 tx=0x7fdae4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecca99ec1f6dd36 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.225+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00a3b0 con 0x7fdae4004d40 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.225+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec034650 con 0x7fdae4004d40 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.225+0000 7fdaf37fe640 1 -- 172.21.15.105:0/433194919 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec034a80 con 0x7fdae4004d40 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.226+0000 7fdafac17640 1 -- 172.21.15.105:0/433194919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 msgr2=0x7fdae40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.226+0000 7fdafac17640 1 --2- 172.21.15.105:0/433194919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdae40051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00a790 tx=0x7fdae4005830 comp rx=0 tx=0).stop 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.226+0000 7fdaf91b4640 1 -- 172.21.15.105:0/433194919 reap_dead start 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.226+0000 7fdafac17640 1 -- 172.21.15.105:0/433194919 shutdown_connections 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.226+0000 7fdafac17640 1 -- 172.21.15.105:0/433194919 >> 172.21.15.105:0/433194919 conn(0x7fdaf40d3ce0 msgr2=0x7fdaf40d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:36.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.226+0000 7fdafac17640 1 -- 172.21.15.105:0/433194919 shutdown_connections 2024-03-08T08:27:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.226+0000 7fdafac17640 1 -- 172.21.15.105:0/433194919 wait complete. 2024-03-08T08:27:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.227+0000 7fdafac17640 1 Processor -- start 2024-03-08T08:27:36.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.227+0000 7fdafac17640 1 -- start start 2024-03-08T08:27:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdafac17640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdaf40a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdafac17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf40cc810 0x7fdaf414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdafac17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4152760 0x7fdaf4152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdafac17640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf406d4f0 con 0x7fdae4004d40 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdafac17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf406d370 con 0x7fdaf40cc810 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdafac17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf406d670 con 0x7fdaf4152760 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdaf40a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdaf40a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58728/0 (socket says 172.21.15.105:58728) 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 -- 172.21.15.105:0/1740781148 learned_addr learned my addr 172.21.15.105:0/1740781148 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf91b4640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4152760 0x7fdaf4152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf3fff640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf40cc810 0x7fdaf414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 -- 172.21.15.105:0/1740781148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4152760 msgr2=0x7fdaf4152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4152760 0x7fdaf4152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 -- 172.21.15.105:0/1740781148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf40cc810 msgr2=0x7fdaf414fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf40cc810 0x7fdaf414fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 -- 172.21.15.105:0/1740781148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf41503e0 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf89b3640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdaf40a16b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdae800f850 tx=0x7fdae800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf17fa640 1 -- 172.21.15.105:0/1740781148 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae800fe40 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf17fa640 1 -- 172.21.15.105:0/1740781148 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae8018070 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf40cd870 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdaf17fa640 1 -- 172.21.15.105:0/1740781148 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae800d6d0 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.228+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf40cd2b0 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.229+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf4075120 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.230+0000 7fdaf17fa640 1 -- 172.21.15.105:0/1740781148 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdae800d870 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.230+0000 7fdaf17fa640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdad806b4d0 0x7fdad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.230+0000 7fdaf3fff640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdad806b4d0 0x7fdad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.230+0000 7fdaf17fa640 1 -- 172.21.15.105:0/1740781148 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdae8089c60 con 0x7fdae4004d40 2024-03-08T08:27:36.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.231+0000 7fdaf3fff640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdad806b4d0 0x7fdad806d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdaf409df60 tx=0x7fdae0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.234+0000 7fdaf17fa640 1 -- 172.21.15.105:0/1740781148 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdae8058e30 con 0x7fdae4004d40 2024-03-08T08:27:36.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.405+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdaf40d4300 con 0x7fdae4004d40 2024-03-08T08:27:36.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.406+0000 7fdaf17fa640 1 -- 172.21.15.105:0/1740781148 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdae8058730 con 0x7fdae4004d40 2024-03-08T08:27:36.408 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:27:36.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.407+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdad806b4d0 msgr2=0x7fdad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.407+0000 7fdafac17640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdad806b4d0 0x7fdad806d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdaf409df60 tx=0x7fdae0008040 comp rx=0 tx=0).stop 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.407+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 msgr2=0x7fdaf40a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.407+0000 7fdafac17640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdaf40a16b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdae800f850 tx=0x7fdae800fd10 comp rx=0 tx=0).stop 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.407+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 shutdown_connections 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.407+0000 7fdafac17640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdad806b4d0 0x7fdad806d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.407+0000 7fdafac17640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4152760 0x7fdaf4152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.407+0000 7fdafac17640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf40cc810 0x7fdaf414fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.408+0000 7fdafac17640 1 --2- 172.21.15.105:0/1740781148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdae4004d40 0x7fdaf40a16b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.408+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 >> 172.21.15.105:0/1740781148 conn(0x7fdaf40d3ce0 msgr2=0x7fdaf40ca190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.408+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 shutdown_connections 2024-03-08T08:27:36.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.408+0000 7fdafac17640 1 -- 172.21.15.105:0/1740781148 wait complete. 2024-03-08T08:27:36.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-08T08:27:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.517+0000 7f0fd22e1640 1 Processor -- start 2024-03-08T08:27:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.517+0000 7f0fd22e1640 1 -- start start 2024-03-08T08:27:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.518+0000 7f0fd22e1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0fcc15df20 0x7f0fcc06c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.518+0000 7f0fd22e1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 0x7f0fcc06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.518+0000 7f0fd22e1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc06f830 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.518+0000 7f0fd22e1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc06b610 con 0x7f0fcc15df20 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.518+0000 7f0fd22e1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc06dbb0 con 0x7f0fcc0b5990 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.518+0000 7f0fcbfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0fcc15df20 0x7f0fcc06c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.518+0000 7f0fd087e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0fcc06b1c0 0x7f0fcc0b3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57888/0 (socket says 172.21.15.105:57888) 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.518+0000 7f0fd087e640 1 -- 172.21.15.105:0/635258995 learned_addr learned my addr 172.21.15.105:0/635258995 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.519+0000 7f0fcb7fe640 1 --2- 172.21.15.105:0/635258995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 0x7f0fcc06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.519+0000 7f0fcbfff640 1 -- 172.21.15.105:0/635258995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 msgr2=0x7f0fcc06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.519+0000 7f0fcbfff640 1 --2- 172.21.15.105:0/635258995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 0x7f0fcc06ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.519+0000 7f0fcbfff640 1 -- 172.21.15.105:0/635258995 >> v1:172.21.15.105:6789/0 conn(0x7f0fcc06b1c0 legacy=0x7f0fcc0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.519+0000 7f0fcbfff640 1 -- 172.21.15.105:0/635258995 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc06fad0 con 0x7f0fcc15df20 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.519+0000 7f0fcbfff640 1 --2- 172.21.15.105:0/635258995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc15df20 0x7f0fcc06c8b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0097a0 tx=0x7f0fbc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47345f97e20f77ed server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.519+0000 7f0fcaffd640 1 -- 172.21.15.105:0/635258995 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0315a0 con 0x7f0fcc15df20 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fcaffd640 1 -- 172.21.15.105:0/635258995 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc037070 con 0x7f0fcc15df20 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fcaffd640 1 -- 172.21.15.105:0/635258995 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc031960 con 0x7f0fcc15df20 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fd22e1640 1 -- 172.21.15.105:0/635258995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc15df20 msgr2=0x7f0fcc06c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/635258995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc15df20 0x7f0fcc06c8b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0097a0 tx=0x7f0fbc02b840 comp rx=0 tx=0).stop 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fd22e1640 1 -- 172.21.15.105:0/635258995 shutdown_connections 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/635258995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 0x7f0fcc06ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/635258995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc15df20 0x7f0fcc06c8b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fd22e1640 1 -- 172.21.15.105:0/635258995 >> 172.21.15.105:0/635258995 conn(0x7f0fcc06bd90 msgr2=0x7f0fcc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:36.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fd22e1640 1 -- 172.21.15.105:0/635258995 shutdown_connections 2024-03-08T08:27:36.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.520+0000 7f0fd22e1640 1 -- 172.21.15.105:0/635258995 wait complete. 2024-03-08T08:27:36.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.521+0000 7f0fd22e1640 1 Processor -- start 2024-03-08T08:27:36.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.521+0000 7f0fd22e1640 1 -- start start 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fd22e1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc06b1c0 0x7f0fcc145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fd22e1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 0x7f0fcc156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fd22e1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fcc15df20 0x7f0fcc0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fd22e1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc06e0e0 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fd22e1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc06de00 con 0x7f0fcc15df20 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fd22e1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0ae980 con 0x7f0fcc0b5990 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fcbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc06b1c0 0x7f0fcc145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fcbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc06b1c0 0x7f0fcc145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58748/0 (socket says 172.21.15.105:58748) 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fcbfff640 1 -- 172.21.15.105:0/2277698619 learned_addr learned my addr 172.21.15.105:0/2277698619 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fcb7fe640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 0x7f0fcc156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fd087e640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fcc15df20 0x7f0fcc0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fcbfff640 1 -- 172.21.15.105:0/2277698619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 msgr2=0x7f0fcc156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fcbfff640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 0x7f0fcc156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fcbfff640 1 -- 172.21.15.105:0/2277698619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fcc15df20 msgr2=0x7f0fcc0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.522+0000 7f0fcbfff640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fcc15df20 0x7f0fcc0c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.523+0000 7f0fcbfff640 1 -- 172.21.15.105:0/2277698619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc156e80 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.523+0000 7f0fcbfff640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc06b1c0 0x7f0fcc145ff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc034040 tx=0x7f0fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.523+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/2277698619 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc032630 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.523+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/2277698619 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc0327d0 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.523+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc1469b0 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.523+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/2277698619 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc032ba0 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.523+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc146f70 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.524+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc075120 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.524+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/2277698619 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fbc04e430 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.524+0000 7f0fc8ff9640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.525+0000 7f0fcb7fe640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.525+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/2277698619 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0fbc02f030 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.525+0000 7f0fcb7fe640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0fb806b4d0 0x7f0fb806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc1561d0 tx=0x7f0fc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.529+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/2277698619 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fbc040070 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.704+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0fcc0b2140 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.704+0000 7f0fc8ff9640 1 -- 172.21.15.105:0/2277698619 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0fbc07af80 con 0x7f0fcc06b1c0 2024-03-08T08:27:36.707 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:36.707 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-03-08T08:27:36.707 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:27:36.707 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-03-08T08:27:36.707 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-03-08T08:27:36.707 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:27:36.707 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-03-08T08:27:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.706+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0fb806b4d0 msgr2=0x7f0fb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.706+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0fb806b4d0 0x7f0fb806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc1561d0 tx=0x7f0fc0008040 comp rx=0 tx=0).stop 2024-03-08T08:27:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.706+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc06b1c0 msgr2=0x7f0fcc145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.706+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc06b1c0 0x7f0fcc145ff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc034040 tx=0x7f0fbc0046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.707+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 shutdown_connections 2024-03-08T08:27:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.707+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.707+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fcc15df20 0x7f0fcc0c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.707+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fcc0b5990 0x7f0fcc156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.707+0000 7f0fd22e1640 1 --2- 172.21.15.105:0/2277698619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fcc06b1c0 0x7f0fcc145ff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.707+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 >> 172.21.15.105:0/2277698619 conn(0x7f0fcc06bd90 msgr2=0x7f0fcc0b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:36.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.707+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 shutdown_connections 2024-03-08T08:27:36.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.707+0000 7f0fd22e1640 1 -- 172.21.15.105:0/2277698619 wait complete. 2024-03-08T08:27:36.719 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-08T08:27:36.719 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-08T08:27:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.820+0000 7f81d2181640 1 Processor -- start 2024-03-08T08:27:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.820+0000 7f81d2181640 1 -- start start 2024-03-08T08:27:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81d2181640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc15e110 0x7f81cc0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81d2181640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f81cc15e4f0 0x7f81cc0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81d2181640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc0cf4a0 con 0x7f81cc0d2140 2024-03-08T08:27:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81d2181640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc0b0b00 con 0x7f81cc15e4f0 2024-03-08T08:27:36.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81d2181640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0cf740 con 0x7f81cc15e110 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81caffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f81cc15e4f0 0x7f81cc0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81cbfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f81cc0d2140 0x7f81cc0d0710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57898/0 (socket says 172.21.15.105:57898) 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81cbfff640 1 -- 172.21.15.105:0/146988503 learned_addr learned my addr 172.21.15.105:0/146988503 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.821+0000 7f81cb7fe640 1 --2- 172.21.15.105:0/146988503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc15e110 0x7f81cc0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81caffd640 1 -- 172.21.15.105:0/146988503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc15e110 msgr2=0x7f81cc0d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81caffd640 1 --2- 172.21.15.105:0/146988503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc15e110 0x7f81cc0d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81caffd640 1 -- 172.21.15.105:0/146988503 >> v1:172.21.15.105:6789/0 conn(0x7f81cc0d2140 legacy=0x7f81cc0d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81ca7fc640 1 -- 172.21.15.105:0/146988503 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832592074 0 0) 0x7f81cc0cf4a0 con 0x7f81cc0d2140 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81caffd640 1 -- 172.21.15.105:0/146988503 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0cfa20 con 0x7f81cc15e4f0 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81caffd640 1 --2- 172.21.15.105:0/146988503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0d1c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f81c00097a0 tx=0x7f81c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1f29bddd60b7a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81ca7fc640 1 -- 172.21.15.105:0/146988503 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00315a0 con 0x7f81cc15e4f0 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81ca7fc640 1 -- 172.21.15.105:0/146988503 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c0037070 con 0x7f81cc15e4f0 2024-03-08T08:27:36.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.822+0000 7f81ca7fc640 1 -- 172.21.15.105:0/146988503 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c0031960 con 0x7f81cc15e4f0 2024-03-08T08:27:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.823+0000 7f81d2181640 1 -- 172.21.15.105:0/146988503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 msgr2=0x7f81cc0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.823+0000 7f81d2181640 1 --2- 172.21.15.105:0/146988503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0d1c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f81c00097a0 tx=0x7f81c002b840 comp rx=0 tx=0).stop 2024-03-08T08:27:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.823+0000 7f81d2181640 1 -- 172.21.15.105:0/146988503 shutdown_connections 2024-03-08T08:27:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.823+0000 7f81d2181640 1 --2- 172.21.15.105:0/146988503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0d1c00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.823+0000 7f81d2181640 1 --2- 172.21.15.105:0/146988503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc15e110 0x7f81cc0d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.823+0000 7f81d2181640 1 -- 172.21.15.105:0/146988503 >> 172.21.15.105:0/146988503 conn(0x7f81cc0d3d00 msgr2=0x7f81cc0d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.823+0000 7f81d2181640 1 -- 172.21.15.105:0/146988503 shutdown_connections 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.823+0000 7f81d2181640 1 -- 172.21.15.105:0/146988503 wait complete. 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.824+0000 7f81d2181640 1 Processor -- start 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.824+0000 7f81d2181640 1 -- start start 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81d2181640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc0d2140 0x7f81cc0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81d2181640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc15e110 0x7f81cc14a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81d2181640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81d2181640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0b2120 con 0x7f81cc15e4f0 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81d2181640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0b1fa0 con 0x7f81cc15e110 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81d2181640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0b22a0 con 0x7f81cc0d2140 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58766/0 (socket says 172.21.15.105:58766) 2024-03-08T08:27:36.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cbfff640 1 -- 172.21.15.105:0/2889249026 learned_addr learned my addr 172.21.15.105:0/2889249026 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cb7fe640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc0d2140 0x7f81cc0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81caffd640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc15e110 0x7f81cc14a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cbfff640 1 -- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc0d2140 msgr2=0x7f81cc0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cbfff640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc0d2140 0x7f81cc0c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cbfff640 1 -- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc15e110 msgr2=0x7f81cc14a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cbfff640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc15e110 0x7f81cc14a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81cbfff640 1 -- 172.21.15.105:0/2889249026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc14acd0 con 0x7f81cc15e4f0 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.825+0000 7f81caffd640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc15e110 0x7f81cc14a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.826+0000 7f81cbfff640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0b2690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81c400c7d0 tx=0x7f81c400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.826+0000 7f81b3fff640 1 -- 172.21.15.105:0/2889249026 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4018070 con 0x7f81cc15e4f0 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.826+0000 7f81b3fff640 1 -- 172.21.15.105:0/2889249026 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c400a640 con 0x7f81cc15e4f0 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.826+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc0c9850 con 0x7f81cc15e4f0 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.826+0000 7f81b3fff640 1 -- 172.21.15.105:0/2889249026 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c400aa70 con 0x7f81cc15e4f0 2024-03-08T08:27:36.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.826+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc0c9e10 con 0x7f81cc15e4f0 2024-03-08T08:27:36.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.827+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc075200 con 0x7f81cc15e4f0 2024-03-08T08:27:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.827+0000 7f81b3fff640 1 -- 172.21.15.105:0/2889249026 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81c401b070 con 0x7f81cc15e4f0 2024-03-08T08:27:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.827+0000 7f81b3fff640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f81b806b4d0 0x7f81b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.828+0000 7f81b3fff640 1 -- 172.21.15.105:0/2889249026 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f81c40893f0 con 0x7f81cc15e4f0 2024-03-08T08:27:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.828+0000 7f81cb7fe640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f81b806b4d0 0x7f81b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:36.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.828+0000 7f81cb7fe640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f81b806b4d0 0x7f81b806d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81bc006fd0 tx=0x7f81bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:36.832+0000 7f81b3fff640 1 -- 172.21.15.105:0/2889249026 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81cc14acd0 con 0x7f81cc15e4f0 2024-03-08T08:27:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.001+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f81cc0746b0 con 0x7f81cc15e4f0 2024-03-08T08:27:37.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.635+0000 7f81b3fff640 1 -- 172.21.15.105:0/2889249026 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f81c4058650 con 0x7f81cc15e4f0 2024-03-08T08:27:37.637 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-03-08T08:27:37.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f81b806b4d0 msgr2=0x7f81b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:37.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f81b806b4d0 0x7f81b806d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81bc006fd0 tx=0x7f81bc008040 comp rx=0 tx=0).stop 2024-03-08T08:27:37.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 msgr2=0x7f81cc0b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0b2690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81c400c7d0 tx=0x7f81c400cc90 comp rx=0 tx=0).stop 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 shutdown_connections 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f81b806b4d0 0x7f81b806d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81cc15e4f0 0x7f81cc0b2690 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81cc15e110 0x7f81cc14a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 --2- 172.21.15.105:0/2889249026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81cc0d2140 0x7f81cc0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 >> 172.21.15.105:0/2889249026 conn(0x7f81cc0d3d00 msgr2=0x7f81cc15d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.638+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 shutdown_connections 2024-03-08T08:27:37.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.639+0000 7f81d2181640 1 -- 172.21.15.105:0/2889249026 wait complete. 2024-03-08T08:27:37.657 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-08T08:27:37.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.765+0000 7f6963265640 1 Processor -- start 2024-03-08T08:27:37.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.766+0000 7f6963265640 1 -- start start 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.766+0000 7f6963265640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f695c144a00 0x7f695c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.766+0000 7f6963265640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c144f60 0x7f695c06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.766+0000 7f6963265640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f695c06c8b0 0x7f695c06b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.766+0000 7f6963265640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c1457b0 con 0x7f695c06c8b0 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.766+0000 7f6963265640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c1454d0 con 0x7f695c144a00 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.766+0000 7f6963265640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c15c2c0 con 0x7f695c144f60 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6960800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c144f60 0x7f695c06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961802640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f695c06c8b0 0x7f695c06b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961802640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f695c06c8b0 0x7f695c06b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58772/0 (socket says 172.21.15.105:58772) 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6960800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c144f60 0x7f695c06ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:54686/0 (socket says 172.21.15.105:54686) 2024-03-08T08:27:37.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961802640 1 -- 172.21.15.105:0/856727709 learned_addr learned my addr 172.21.15.105:0/856727709 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:37.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961001640 1 --2- 172.21.15.105:0/856727709 >> v2:172.21.15.144:3300/0 conn(0x7f695c144a00 0x7f695c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:37.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961802640 1 -- 172.21.15.105:0/856727709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c144f60 msgr2=0x7f695c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:37.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961802640 1 --2- 172.21.15.105:0/856727709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c144f60 0x7f695c06ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961802640 1 -- 172.21.15.105:0/856727709 >> v2:172.21.15.144:3300/0 conn(0x7f695c144a00 msgr2=0x7f695c15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:37.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961802640 1 --2- 172.21.15.105:0/856727709 >> v2:172.21.15.144:3300/0 conn(0x7f695c144a00 0x7f695c15cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961802640 1 -- 172.21.15.105:0/856727709 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c15c5a0 con 0x7f695c06c8b0 2024-03-08T08:27:37.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.767+0000 7f6961001640 1 --2- 172.21.15.105:0/856727709 >> v2:172.21.15.144:3300/0 conn(0x7f695c144a00 0x7f695c15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:37.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.768+0000 7f6961802640 1 --2- 172.21.15.105:0/856727709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c06c8b0 0x7f695c06b1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f695c074ad0 tx=0x7f695402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f63e46b10c5268e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:37.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.768+0000 7f6953fff640 1 -- 172.21.15.105:0/856727709 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954035020 con 0x7f695c06c8b0 2024-03-08T08:27:37.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.768+0000 7f6953fff640 1 -- 172.21.15.105:0/856727709 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954031870 con 0x7f695c06c8b0 2024-03-08T08:27:37.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.768+0000 7f6953fff640 1 -- 172.21.15.105:0/856727709 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954031c40 con 0x7f695c06c8b0 2024-03-08T08:27:37.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.768+0000 7f6963265640 1 -- 172.21.15.105:0/856727709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c06c8b0 msgr2=0x7f695c06b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:37.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.768+0000 7f6963265640 1 --2- 172.21.15.105:0/856727709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c06c8b0 0x7f695c06b1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f695c074ad0 tx=0x7f695402b910 comp rx=0 tx=0).stop 2024-03-08T08:27:37.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.769+0000 7f6963265640 1 -- 172.21.15.105:0/856727709 shutdown_connections 2024-03-08T08:27:37.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.769+0000 7f6963265640 1 --2- 172.21.15.105:0/856727709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c06c8b0 0x7f695c06b1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.769+0000 7f6963265640 1 --2- 172.21.15.105:0/856727709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c144f60 0x7f695c06ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.769+0000 7f6963265640 1 --2- 172.21.15.105:0/856727709 >> v2:172.21.15.144:3300/0 conn(0x7f695c144a00 0x7f695c15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.769+0000 7f6963265640 1 -- 172.21.15.105:0/856727709 >> 172.21.15.105:0/856727709 conn(0x7f695c06bd90 msgr2=0x7f695c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:37.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.769+0000 7f6963265640 1 -- 172.21.15.105:0/856727709 shutdown_connections 2024-03-08T08:27:37.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.769+0000 7f6963265640 1 -- 172.21.15.105:0/856727709 wait complete. 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.770+0000 7f6963265640 1 Processor -- start 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.770+0000 7f6963265640 1 -- start start 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6963265640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c06c8b0 0x7f695c160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6963265640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695c144a00 0x7f695c1f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6963265640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c144f60 0x7f695c1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6963265640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c06ff10 con 0x7f695c144f60 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6963265640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c15d2e0 con 0x7f695c144a00 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6963265640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c0cee70 con 0x7f695c06c8b0 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c144f60 0x7f695c1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c144f60 0x7f695c1f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58782/0 (socket says 172.21.15.105:58782) 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 -- 172.21.15.105:0/2321745564 learned_addr learned my addr 172.21.15.105:0/2321745564 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961001640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c06c8b0 0x7f695c160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6960800640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695c144a00 0x7f695c1f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 -- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c06c8b0 msgr2=0x7f695c160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c06c8b0 0x7f695c160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 -- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695c144a00 msgr2=0x7f695c1f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695c144a00 0x7f695c1f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 -- 172.21.15.105:0/2321745564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c06ee30 con 0x7f695c144f60 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961001640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c06c8b0 0x7f695c160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6961802640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c144f60 0x7f695c1f2440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f695402b950 tx=0x7f69540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6951ffb640 1 -- 172.21.15.105:0/2321745564 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954040070 con 0x7f695c144f60 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6951ffb640 1 -- 172.21.15.105:0/2321745564 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954030d00 con 0x7f695c144f60 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6951ffb640 1 -- 172.21.15.105:0/2321745564 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954035040 con 0x7f695c144f60 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.771+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695c15f980 con 0x7f695c144f60 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.772+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c06e590 con 0x7f695c144f60 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.772+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c075120 con 0x7f695c144f60 2024-03-08T08:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.773+0000 7f6951ffb640 1 -- 172.21.15.105:0/2321745564 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6954038020 con 0x7f695c144f60 2024-03-08T08:27:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.773+0000 7f6951ffb640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f694006b4d0 0x7f694006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.774+0000 7f6961001640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f694006b4d0 0x7f694006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.774+0000 7f6951ffb640 1 -- 172.21.15.105:0/2321745564 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f69540abde0 con 0x7f695c144f60 2024-03-08T08:27:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.774+0000 7f6961001640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f694006b4d0 0x7f694006d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6958009ac0 tx=0x7f69580039b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:37.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.778+0000 7f6951ffb640 1 -- 172.21.15.105:0/2321745564 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f695407ae40 con 0x7f695c144f60 2024-03-08T08:27:37.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:37.951+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f695c15bca0 con 0x7f695c144f60 2024-03-08T08:27:38.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.638+0000 7f6951ffb640 1 -- 172.21.15.105:0/2321745564 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f695407a740 con 0x7f695c144f60 2024-03-08T08:27:38.640 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-03-08T08:27:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.641+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f694006b4d0 msgr2=0x7f694006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.641+0000 7f6963265640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f694006b4d0 0x7f694006d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6958009ac0 tx=0x7f69580039b0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c144f60 msgr2=0x7f695c1f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c144f60 0x7f695c1f2440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f695402b950 tx=0x7f69540046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 shutdown_connections 2024-03-08T08:27:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f694006b4d0 0x7f694006d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f695c144f60 0x7f695c1f2440 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f695c144a00 0x7f695c1f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 --2- 172.21.15.105:0/2321745564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f695c06c8b0 0x7f695c160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 >> 172.21.15.105:0/2321745564 conn(0x7f695c06bd90 msgr2=0x7f695c0720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.642+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 shutdown_connections 2024-03-08T08:27:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.643+0000 7f6963265640 1 -- 172.21.15.105:0/2321745564 wait complete. 2024-03-08T08:27:38.670 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-08T08:27:38.694 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:27:38.716 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:27:38.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.777+0000 7f4910ec3640 1 Processor -- start 2024-03-08T08:27:38.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.777+0000 7f4910ec3640 1 -- start start 2024-03-08T08:27:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.777+0000 7f4910ec3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f4910ec3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f490c0d07c0 0x7f490c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f4910ec3640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c15e720 con 0x7f490c066990 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f4910ec3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c15f020 con 0x7f490c0d07c0 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f4910ec3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c066ec0 con 0x7f490c15e340 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f490ad9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f490c0d07c0 0x7f490c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f490a59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f4903fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f490c066990 0x7f490c0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:53194/0 (socket says 172.21.15.105:53194) 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f4903fff640 1 -- 172.21.15.105:0/1957029499 learned_addr learned my addr 172.21.15.105:0/1957029499 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f490a59c640 1 -- 172.21.15.105:0/1957029499 >> v2:172.21.15.144:3300/0 conn(0x7f490c0d07c0 msgr2=0x7f490c0b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f490a59c640 1 --2- 172.21.15.105:0/1957029499 >> v2:172.21.15.144:3300/0 conn(0x7f490c0d07c0 0x7f490c0b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f490a59c640 1 -- 172.21.15.105:0/1957029499 >> v1:172.21.15.144:6789/0 conn(0x7f490c066990 legacy=0x7f490c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f490a59c640 1 -- 172.21.15.105:0/1957029499 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c15e9d0 con 0x7f490c15e340 2024-03-08T08:27:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.778+0000 7f490ad9d640 1 --2- 172.21.15.105:0/1957029499 >> v2:172.21.15.144:3300/0 conn(0x7f490c0d07c0 0x7f490c0b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.779+0000 7f490a59c640 1 --2- 172.21.15.105:0/1957029499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c06b690 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0097a0 tx=0x7f48fc02b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8cc818c916c2974 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.779+0000 7f4909d9b640 1 -- 172.21.15.105:0/1957029499 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc037070 con 0x7f490c15e340 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.779+0000 7f4909d9b640 1 -- 172.21.15.105:0/1957029499 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc030710 con 0x7f490c15e340 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.779+0000 7f4909d9b640 1 -- 172.21.15.105:0/1957029499 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc030ae0 con 0x7f490c15e340 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.780+0000 7f4910ec3640 1 -- 172.21.15.105:0/1957029499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 msgr2=0x7f490c06b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.780+0000 7f4910ec3640 1 --2- 172.21.15.105:0/1957029499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c06b690 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0097a0 tx=0x7f48fc02b5d0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.780+0000 7f4910ec3640 1 -- 172.21.15.105:0/1957029499 shutdown_connections 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.780+0000 7f4910ec3640 1 --2- 172.21.15.105:0/1957029499 >> v2:172.21.15.144:3300/0 conn(0x7f490c0d07c0 0x7f490c0b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.780+0000 7f4910ec3640 1 --2- 172.21.15.105:0/1957029499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c06b690 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.780+0000 7f4910ec3640 1 -- 172.21.15.105:0/1957029499 >> 172.21.15.105:0/1957029499 conn(0x7f490c072110 msgr2=0x7f490c072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.780+0000 7f4910ec3640 1 -- 172.21.15.105:0/1957029499 shutdown_connections 2024-03-08T08:27:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.780+0000 7f4910ec3640 1 -- 172.21.15.105:0/1957029499 wait complete. 2024-03-08T08:27:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.781+0000 7f4910ec3640 1 Processor -- start 2024-03-08T08:27:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.781+0000 7f4910ec3640 1 -- start start 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4910ec3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c066990 0x7f490c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4910ec3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f490c0d07c0 0x7f490c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4910ec3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c0cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4910ec3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c15f4b0 con 0x7f490c0d07c0 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4910ec3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0af530 con 0x7f490c066990 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4910ec3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c158310 con 0x7f490c15e340 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f490c0d07c0 0x7f490c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f490a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c066990 0x7f490c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f490c0d07c0 0x7f490c0cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58790/0 (socket says 172.21.15.105:58790) 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 -- 172.21.15.105:0/3545102758 learned_addr learned my addr 172.21.15.105:0/3545102758 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f490ad9d640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c0cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 -- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 msgr2=0x7f490c0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c0cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 -- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c066990 msgr2=0x7f490c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c066990 0x7f490c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 -- 172.21.15.105:0/3545102758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c1612f0 con 0x7f490c0d07c0 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f490ad9d640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c0cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f490a59c640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c066990 0x7f490c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.782+0000 7f4903fff640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f490c0d07c0 0x7f490c0cef50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48f400c570 tx=0x7f48f400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.783+0000 7f4902ffd640 1 -- 172.21.15.105:0/3545102758 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4018070 con 0x7f490c0d07c0 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.783+0000 7f4902ffd640 1 -- 172.21.15.105:0/3545102758 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f40074e0 con 0x7f490c0d07c0 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.783+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c1585b0 con 0x7f490c0d07c0 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.783+0000 7f4902ffd640 1 -- 172.21.15.105:0/3545102758 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4004a20 con 0x7f490c0d07c0 2024-03-08T08:27:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.783+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c161590 con 0x7f490c0d07c0 2024-03-08T08:27:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.783+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c161790 con 0x7f490c0d07c0 2024-03-08T08:27:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.784+0000 7f4902ffd640 1 -- 172.21.15.105:0/3545102758 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48f401b070 con 0x7f490c0d07c0 2024-03-08T08:27:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.784+0000 7f4902ffd640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f48ec06b500 0x7f48ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.785+0000 7f4902ffd640 1 -- 172.21.15.105:0/3545102758 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f48f40893a0 con 0x7f490c0d07c0 2024-03-08T08:27:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.785+0000 7f490a59c640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f48ec06b500 0x7f48ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.785+0000 7f490a59c640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f48ec06b500 0x7f48ec06d9c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48fc034000 tx=0x7f48fc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:38.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.789+0000 7f4902ffd640 1 -- 172.21.15.105:0/3545102758 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f490c161790 con 0x7f490c0d07c0 2024-03-08T08:27:38.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:38.996+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f490c072e90 con 0x7f490c0d07c0 2024-03-08T08:27:39.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.659+0000 7f4902ffd640 1 -- 172.21.15.105:0/3545102758 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f48f4001210 con 0x7f490c0d07c0 2024-03-08T08:27:39.661 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 2 and data pool 3 2024-03-08T08:27:39.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f48ec06b500 msgr2=0x7f48ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:39.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f48ec06b500 0x7f48ec06d9c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48fc034000 tx=0x7f48fc008030 comp rx=0 tx=0).stop 2024-03-08T08:27:39.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f490c0d07c0 msgr2=0x7f490c0cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:39.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f490c0d07c0 0x7f490c0cef50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48f400c570 tx=0x7f48f400c970 comp rx=0 tx=0).stop 2024-03-08T08:27:39.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 shutdown_connections 2024-03-08T08:27:39.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f48ec06b500 0x7f48ec06d9c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f490c15e340 0x7f490c0cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f490c0d07c0 0x7f490c0cef50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 --2- 172.21.15.105:0/3545102758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c066990 0x7f490c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.664+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 >> 172.21.15.105:0/3545102758 conn(0x7f490c072110 msgr2=0x7f490c0b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:39.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.665+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 shutdown_connections 2024-03-08T08:27:39.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.666+0000 7f4910ec3640 1 -- 172.21.15.105:0/3545102758 wait complete. 2024-03-08T08:27:39.689 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:39.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.829+0000 7f581b076640 1 Processor -- start 2024-03-08T08:27:39.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.829+0000 7f581b076640 1 -- start start 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.829+0000 7f581b076640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58141453d0 0x7f58140cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.829+0000 7f581b076640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58140cd220 0x7f581406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.829+0000 7f581b076640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58141457b0 con 0x7f58140cf340 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.829+0000 7f581b076640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140b44e0 con 0x7f58141453d0 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.829+0000 7f581b076640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58140b47c0 con 0x7f58140cd220 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.830+0000 7f5818e12640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58141453d0 0x7f58140cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.830+0000 7f5813fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f58140cf340 0x7f58140ccb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57914/0 (socket says 172.21.15.105:57914) 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.830+0000 7f5813fff640 1 -- 172.21.15.105:0/620573122 learned_addr learned my addr 172.21.15.105:0/620573122 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:39.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.830+0000 7f5818e12640 1 -- 172.21.15.105:0/620573122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58140cd220 msgr2=0x7f581406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:39.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.830+0000 7f5818e12640 1 --2- 172.21.15.105:0/620573122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58140cd220 0x7f581406cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.830+0000 7f5818e12640 1 -- 172.21.15.105:0/620573122 >> v1:172.21.15.105:6789/0 conn(0x7f58140cf340 legacy=0x7f58140ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:39.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.830+0000 7f5818e12640 1 -- 172.21.15.105:0/620573122 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140b4aa0 con 0x7f58141453d0 2024-03-08T08:27:39.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.830+0000 7f58137fe640 1 -- 172.21.15.105:0/620573122 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978524627 0 0) 0x7f58141457b0 con 0x7f58140cf340 2024-03-08T08:27:39.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f5818e12640 1 --2- 172.21.15.105:0/620573122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58141453d0 0x7f58140cee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f58080097a0 tx=0x7f580802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38e4ba9f7dd46a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:39.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f58137fe640 1 -- 172.21.15.105:0/620573122 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808035020 con 0x7f58141453d0 2024-03-08T08:27:39.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f58137fe640 1 -- 172.21.15.105:0/620573122 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5808031830 con 0x7f58141453d0 2024-03-08T08:27:39.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f58137fe640 1 -- 172.21.15.105:0/620573122 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808031c00 con 0x7f58141453d0 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f581b076640 1 -- 172.21.15.105:0/620573122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58141453d0 msgr2=0x7f58140cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f581b076640 1 --2- 172.21.15.105:0/620573122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58141453d0 0x7f58140cee00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f58080097a0 tx=0x7f580802b8d0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f581b076640 1 -- 172.21.15.105:0/620573122 shutdown_connections 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f581b076640 1 --2- 172.21.15.105:0/620573122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58140cd220 0x7f581406cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f581b076640 1 --2- 172.21.15.105:0/620573122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58141453d0 0x7f58140cee00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f581b076640 1 -- 172.21.15.105:0/620573122 >> 172.21.15.105:0/620573122 conn(0x7f58140b1740 msgr2=0x7f58140b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f581b076640 1 -- 172.21.15.105:0/620573122 shutdown_connections 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.831+0000 7f581b076640 1 -- 172.21.15.105:0/620573122 wait complete. 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.832+0000 7f581b076640 1 Processor -- start 2024-03-08T08:27:39.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.832+0000 7f581b076640 1 -- start start 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f581b076640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58140cd220 0x7f581415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f581b076640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58140cf340 0x7f58141e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f581b076640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58141453d0 0x7f5814142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f581b076640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581406d000 con 0x7f58140cf340 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f581b076640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58140b4190 con 0x7f58140cd220 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f581b076640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581406d180 con 0x7f58141453d0 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f5813fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58140cf340 0x7f58141e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f5813fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58140cf340 0x7f58141e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58804/0 (socket says 172.21.15.105:58804) 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.833+0000 7f5813fff640 1 -- 172.21.15.105:0/4281953046 learned_addr learned my addr 172.21.15.105:0/4281953046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5819613640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58141453d0 0x7f5814142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5818e12640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58140cd220 0x7f581415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:39.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5813fff640 1 -- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58141453d0 msgr2=0x7f5814142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5813fff640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58141453d0 0x7f5814142480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5813fff640 1 -- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58140cd220 msgr2=0x7f581415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5813fff640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58140cd220 0x7f581415afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5813fff640 1 -- 172.21.15.105:0/4281953046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5814070740 con 0x7f58140cf340 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5819613640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58141453d0 0x7f5814142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f5813fff640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58140cf340 0x7f58141e42d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f580000c4d0 tx=0x7f580000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f58117fa640 1 -- 172.21.15.105:0/4281953046 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800018070 con 0x7f58140cf340 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f58117fa640 1 -- 172.21.15.105:0/4281953046 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5800007500 con 0x7f58140cf340 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f58117fa640 1 -- 172.21.15.105:0/4281953046 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800004a20 con 0x7f58140cf340 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5814070a20 con 0x7f58140cf340 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.834+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5814070fe0 con 0x7f58140cf340 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.835+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58140711e0 con 0x7f58140cf340 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.835+0000 7f58117fa640 1 -- 172.21.15.105:0/4281953046 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f580001b070 con 0x7f58140cf340 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.836+0000 7f58117fa640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f57f806b4d0 0x7f57f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:39.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.836+0000 7f58117fa640 1 -- 172.21.15.105:0/4281953046 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5800089510 con 0x7f58140cf340 2024-03-08T08:27:39.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.836+0000 7f5818e12640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f57f806b4d0 0x7f57f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:39.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.836+0000 7f5818e12640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f57f806b4d0 0x7f57f806d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5808036000 tx=0x7f580803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:39.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:39.840+0000 7f58117fa640 1 -- 172.21.15.105:0/4281953046 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58140711e0 con 0x7f58140cf340 2024-03-08T08:27:40.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.011+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5814075120 con 0x7f58140cf340 2024-03-08T08:27:40.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.012+0000 7f58117fa640 1 -- 172.21.15.105:0/4281953046 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16630 (secure 0 0 0) 0x7f5800058390 con 0x7f58140cf340 2024-03-08T08:27:40.014 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:40.015 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":19,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:38.998048+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:40.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f57f806b4d0 msgr2=0x7f57f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f57f806b4d0 0x7f57f806d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5808036000 tx=0x7f580803c040 comp rx=0 tx=0).stop 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58140cf340 msgr2=0x7f58141e42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58140cf340 0x7f58141e42d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f580000c4d0 tx=0x7f580000c990 comp rx=0 tx=0).stop 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 shutdown_connections 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f57f806b4d0 0x7f57f806d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58141453d0 0x7f5814142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58140cf340 0x7f58141e42d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 --2- 172.21.15.105:0/4281953046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58140cd220 0x7f581415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 >> 172.21.15.105:0/4281953046 conn(0x7f58140b1740 msgr2=0x7f581415c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 shutdown_connections 2024-03-08T08:27:40.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.014+0000 7f581b076640 1 -- 172.21.15.105:0/4281953046 wait complete. 2024-03-08T08:27:40.027 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.125+0000 7f5f3db5e640 1 Processor -- start 2024-03-08T08:27:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.125+0000 7f5f3db5e640 1 -- start start 2024-03-08T08:27:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f3db5e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5f380d2530 0x7f5f380b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f3db5e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380d48f0 con 0x7f5f380d43e0 2024-03-08T08:27:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f3db5e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380d4bb0 con 0x7f5f380d2910 2024-03-08T08:27:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f3db5e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f380ceb30 con 0x7f5f380d2530 2024-03-08T08:27:40.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f36ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5f380d43e0 0x7f5f380b3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57922/0 (socket says 172.21.15.105:57922) 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f36ffd640 1 -- 172.21.15.105:0/2201752402 learned_addr learned my addr 172.21.15.105:0/2201752402 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802569254 0 0) 0x7f5f380d48f0 con 0x7f5f380d43e0 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f377fe640 1 --2- 172.21.15.105:0/2201752402 >> v2:172.21.15.144:3300/0 conn(0x7f5f380d2530 0x7f5f380b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f20003520 con 0x7f5f380d43e0 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2679084518 0 0) 0x7f5f20003520 con 0x7f5f380d43e0 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 >> v2:172.21.15.144:3300/0 conn(0x7f5f380d2530 msgr2=0x7f5f380b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f367fc640 1 --2- 172.21.15.105:0/2201752402 >> v2:172.21.15.144:3300/0 conn(0x7f5f380d2530 0x7f5f380b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.126+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 >> v1:172.21.15.144:6789/0 conn(0x7f5f380d2910 legacy=0x7f5f3815de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380cee10 con 0x7f5f380d43e0 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f5f2c001670 con 0x7f5f380d43e0 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f377fe640 1 --2- 172.21.15.105:0/2201752402 >> v2:172.21.15.144:3300/0 conn(0x7f5f380d2530 0x7f5f380b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 >> v1:172.21.15.105:6789/0 conn(0x7f5f380d43e0 legacy=0x7f5f380b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f200039b0 0x7f5f20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f20004800 0x7f5f20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f200052e0 0x7f5f2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380ceb30 con 0x7f5f200052e0 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2000c1b0 con 0x7f5f200039b0 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2000c3b0 con 0x7f5f20004800 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f37fff640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f200052e0 0x7f5f2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f377fe640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f200039b0 0x7f5f20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f36ffd640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f20004800 0x7f5f20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f37fff640 1 -- 172.21.15.105:0/2201752402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f20004800 msgr2=0x7f5f20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f37fff640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f20004800 0x7f5f20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f37fff640 1 -- 172.21.15.105:0/2201752402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f200039b0 msgr2=0x7f5f20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f37fff640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f200039b0 0x7f5f20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.127+0000 7f5f37fff640 1 -- 172.21.15.105:0/2201752402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20003520 con 0x7f5f200052e0 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f37fff640 1 -- 172.21.15.105:0/2201752402 reap_dead start 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f37fff640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f200052e0 0x7f5f2000b9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5f30001e20 tx=0x7f5f20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06c4fe6ec350fee server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30001a70 con 0x7f5f200052e0 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30003b70 con 0x7f5f200052e0 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f367fc640 1 -- 172.21.15.105:0/2201752402 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300050d0 con 0x7f5f200052e0 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f3db5e640 1 -- 172.21.15.105:0/2201752402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f200052e0 msgr2=0x7f5f2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f3db5e640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f200052e0 0x7f5f2000b9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5f30001e20 tx=0x7f5f20005830 comp rx=0 tx=0).stop 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f3db5e640 1 -- 172.21.15.105:0/2201752402 shutdown_connections 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f3db5e640 1 --2- 172.21.15.105:0/2201752402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f200052e0 0x7f5f2000b9c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f3db5e640 1 -- 172.21.15.105:0/2201752402 >> 172.21.15.105:0/2201752402 conn(0x7f5f3809cfb0 msgr2=0x7f5f380d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f3db5e640 1 -- 172.21.15.105:0/2201752402 shutdown_connections 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.128+0000 7f5f3db5e640 1 -- 172.21.15.105:0/2201752402 wait complete. 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.129+0000 7f5f3db5e640 1 Processor -- start 2024-03-08T08:27:40.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f3db5e640 1 -- start start 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f3db5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f200052e0 0x7f5f3814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f3db5e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f380b7940 0x7f5f380ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f3db5e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f380b8f50 0x7f5f380c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f3db5e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380b5170 con 0x7f5f380b7940 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f3db5e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380b3080 con 0x7f5f380b8f50 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f3db5e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380b52f0 con 0x7f5f200052e0 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f36ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f380b7940 0x7f5f380ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f36ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f380b7940 0x7f5f380ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58808/0 (socket says 172.21.15.105:58808) 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.130+0000 7f5f36ffd640 1 -- 172.21.15.105:0/1755913773 learned_addr learned my addr 172.21.15.105:0/1755913773 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f36ffd640 1 -- 172.21.15.105:0/1755913773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f200052e0 msgr2=0x7f5f3814e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f36ffd640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f200052e0 0x7f5f3814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f36ffd640 1 -- 172.21.15.105:0/1755913773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f380b8f50 msgr2=0x7f5f380c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f36ffd640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f380b8f50 0x7f5f380c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f36ffd640 1 -- 172.21.15.105:0/1755913773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380b7eb0 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f36ffd640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f380b7940 0x7f5f380ca500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c001e80 tx=0x7f5f2c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f1ffff640 1 -- 172.21.15.105:0/1755913773 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00f960 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f1ffff640 1 -- 172.21.15.105:0/1755913773 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c018070 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f1ffff640 1 -- 172.21.15.105:0/1755913773 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00fd80 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f380b8190 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.131+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f380b8750 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.132+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f38075120 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.132+0000 7f5f1ffff640 1 -- 172.21.15.105:0/1755913773 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f2c01d460 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.133+0000 7f5f1ffff640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f2406b4d0 0x7f5f2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.133+0000 7f5f377fe640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f2406b4d0 0x7f5f2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.133+0000 7f5f1ffff640 1 -- 172.21.15.105:0/1755913773 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f2c08a250 con 0x7f5f380b7940 2024-03-08T08:27:40.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.133+0000 7f5f377fe640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f2406b4d0 0x7f5f2406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f28006fa0 tx=0x7f5f28006f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.137+0000 7f5f1ffff640 1 -- 172.21.15.105:0/1755913773 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f2c0590d0 con 0x7f5f380b7940 2024-03-08T08:27:40.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.308+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f3808dd60 con 0x7f5f380b7940 2024-03-08T08:27:40.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.309+0000 7f5f1ffff640 1 -- 172.21.15.105:0/1755913773 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16630 (secure 0 0 0) 0x7f5f2c0589d0 con 0x7f5f380b7940 2024-03-08T08:27:40.312 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:40.312 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":19,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:38.998048+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:40.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f2406b4d0 msgr2=0x7f5f2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f2406b4d0 0x7f5f2406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f28006fa0 tx=0x7f5f28006f10 comp rx=0 tx=0).stop 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f380b7940 msgr2=0x7f5f380ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f380b7940 0x7f5f380ca500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c001e80 tx=0x7f5f2c007720 comp rx=0 tx=0).stop 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 shutdown_connections 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f2406b4d0 0x7f5f2406d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f380b8f50 0x7f5f380c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f380b7940 0x7f5f380ca500 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 --2- 172.21.15.105:0/1755913773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f200052e0 0x7f5f3814e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 >> 172.21.15.105:0/1755913773 conn(0x7f5f3809cfb0 msgr2=0x7f5f380b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 shutdown_connections 2024-03-08T08:27:40.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.311+0000 7f5f3db5e640 1 -- 172.21.15.105:0/1755913773 wait complete. 2024-03-08T08:27:40.324 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-08T08:27:40.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.424+0000 7fb094d1e640 1 Processor -- start 2024-03-08T08:27:40.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.424+0000 7fb094d1e640 1 -- start start 2024-03-08T08:27:40.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.425+0000 7fb094d1e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb09015df20 0x7fb09006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.425+0000 7fb094d1e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09006b390 0x7fb0900ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.425+0000 7fb094d1e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09006f830 con 0x7fb0900b5990 2024-03-08T08:27:40.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.425+0000 7fb094d1e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09006dbb0 con 0x7fb09015df20 2024-03-08T08:27:40.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.425+0000 7fb094d1e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09006fad0 con 0x7fb09006b390 2024-03-08T08:27:40.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.425+0000 7fb08e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb09015df20 0x7fb09006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.425+0000 7fb08e59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb09015df20 0x7fb09006c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58812/0 (socket says 172.21.15.105:58812) 2024-03-08T08:27:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.425+0000 7fb08e59c640 1 -- 172.21.15.105:0/1346028175 learned_addr learned my addr 172.21.15.105:0/1346028175 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.426+0000 7fb08ed9d640 1 --2- 172.21.15.105:0/1346028175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09006b390 0x7fb0900ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.426+0000 7fb08e59c640 1 -- 172.21.15.105:0/1346028175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09006b390 msgr2=0x7fb0900ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.426+0000 7fb08e59c640 1 --2- 172.21.15.105:0/1346028175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09006b390 0x7fb0900ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.426+0000 7fb08e59c640 1 -- 172.21.15.105:0/1346028175 >> v1:172.21.15.144:6789/0 conn(0x7fb0900b5990 legacy=0x7fb09006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.426+0000 7fb08e59c640 1 -- 172.21.15.105:0/1346028175 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09006fdb0 con 0x7fb09015df20 2024-03-08T08:27:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.426+0000 7fb08e59c640 1 --2- 172.21.15.105:0/1346028175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09015df20 0x7fb09006c8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb080009e50 tx=0x7fb08002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8838b0de66bbf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb08d59a640 1 -- 172.21.15.105:0/1346028175 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08002ca70 con 0x7fb09015df20 2024-03-08T08:27:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb08d59a640 1 -- 172.21.15.105:0/1346028175 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08002cc10 con 0x7fb09015df20 2024-03-08T08:27:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb08d59a640 1 -- 172.21.15.105:0/1346028175 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800343f0 con 0x7fb09015df20 2024-03-08T08:27:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb094d1e640 1 -- 172.21.15.105:0/1346028175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09015df20 msgr2=0x7fb09006c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb094d1e640 1 --2- 172.21.15.105:0/1346028175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09015df20 0x7fb09006c8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb080009e50 tx=0x7fb08002bed0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb094d1e640 1 -- 172.21.15.105:0/1346028175 shutdown_connections 2024-03-08T08:27:40.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb094d1e640 1 --2- 172.21.15.105:0/1346028175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09006b390 0x7fb0900ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb094d1e640 1 --2- 172.21.15.105:0/1346028175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09015df20 0x7fb09006c8b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb094d1e640 1 -- 172.21.15.105:0/1346028175 >> 172.21.15.105:0/1346028175 conn(0x7fb09006bd90 msgr2=0x7fb09006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:40.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb094d1e640 1 -- 172.21.15.105:0/1346028175 shutdown_connections 2024-03-08T08:27:40.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.427+0000 7fb094d1e640 1 -- 172.21.15.105:0/1346028175 wait complete. 2024-03-08T08:27:40.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.428+0000 7fb094d1e640 1 Processor -- start 2024-03-08T08:27:40.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb094d1e640 1 -- start start 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb094d1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09006b390 0x7fb090154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb094d1e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0900b5990 0x7fb0900b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb094d1e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09015df20 0x7fb0900b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb094d1e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900adeb0 con 0x7fb09006b390 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb094d1e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09006e0e0 con 0x7fb0900b5990 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb094d1e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900ae030 con 0x7fb09015df20 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb08e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09006b390 0x7fb090154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb08e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09006b390 0x7fb090154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58814/0 (socket says 172.21.15.105:58814) 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.429+0000 7fb08e59c640 1 -- 172.21.15.105:0/737138642 learned_addr learned my addr 172.21.15.105:0/737138642 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08dd9b640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0900b5990 0x7fb0900b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08ed9d640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09015df20 0x7fb0900b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08e59c640 1 -- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09015df20 msgr2=0x7fb0900b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08e59c640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09015df20 0x7fb0900b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08e59c640 1 -- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0900b5990 msgr2=0x7fb0900b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08e59c640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0900b5990 0x7fb0900b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08e59c640 1 -- 172.21.15.105:0/737138642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090154b40 con 0x7fb09006b390 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08dd9b640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0900b5990 0x7fb0900b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb08e59c640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09006b390 0x7fb090154600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb080000f80 tx=0x7fb080004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb07effd640 1 -- 172.21.15.105:0/737138642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080043020 con 0x7fb09006b390 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb07effd640 1 -- 172.21.15.105:0/737138642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08003e450 con 0x7fb09006b390 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb07effd640 1 -- 172.21.15.105:0/737138642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080009510 con 0x7fb09006b390 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09015f260 con 0x7fb09006b390 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.430+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09015f820 con 0x7fb09006b390 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.431+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb090075120 con 0x7fb09006b390 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.431+0000 7fb07effd640 1 -- 172.21.15.105:0/737138642 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb080046020 con 0x7fb09006b390 2024-03-08T08:27:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.432+0000 7fb07effd640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb07006b4d0 0x7fb07006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.432+0000 7fb07effd640 1 -- 172.21.15.105:0/737138642 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0800ac9c0 con 0x7fb09006b390 2024-03-08T08:27:40.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.432+0000 7fb08dd9b640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb07006b4d0 0x7fb07006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.432+0000 7fb08dd9b640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb07006b4d0 0x7fb07006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb090161380 tx=0x7fb084009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.436+0000 7fb07effd640 1 -- 172.21.15.105:0/737138642 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0800aee20 con 0x7fb09006b390 2024-03-08T08:27:40.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.642+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb09015b320 con 0x7fb09006b390 2024-03-08T08:27:40.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.664+0000 7fb07effd640 1 -- 172.21.15.105:0/737138642 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb08004d030 con 0x7fb09006b390 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb07006b4d0 msgr2=0x7fb07006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb07006b4d0 0x7fb07006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb090161380 tx=0x7fb084009040 comp rx=0 tx=0).stop 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09006b390 msgr2=0x7fb090154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09006b390 0x7fb090154600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb080000f80 tx=0x7fb080004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 shutdown_connections 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb07006b4d0 0x7fb07006d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09015df20 0x7fb0900b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0900b5990 0x7fb0900b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 --2- 172.21.15.105:0/737138642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb09006b390 0x7fb090154600 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 >> 172.21.15.105:0/737138642 conn(0x7fb09006bd90 msgr2=0x7fb09006c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:40.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 shutdown_connections 2024-03-08T08:27:40.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.667+0000 7fb094d1e640 1 -- 172.21.15.105:0/737138642 wait complete. 2024-03-08T08:27:40.679 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-08T08:27:40.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-08T08:27:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.780+0000 7fb3e9dc9640 1 Processor -- start 2024-03-08T08:27:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.780+0000 7fb3e9dc9640 1 -- start start 2024-03-08T08:27:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.780+0000 7fb3e9dc9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb3e415bea0 0x7fb3e40b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.780+0000 7fb3e9dc9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415df30 0x7fb3e40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e9dc9640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e415f7b0 con 0x7fb3e40b5a60 2024-03-08T08:27:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e9dc9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e415e710 con 0x7fb3e415bea0 2024-03-08T08:27:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e9dc9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e415e9f0 con 0x7fb3e415df30 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e2ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415df30 0x7fb3e40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e37fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb3e415bea0 0x7fb3e40b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e2ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415df30 0x7fb3e40b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:54782/0 (socket says 172.21.15.105:54782) 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e3fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb3e40b5a60 0x7fb3e409baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:53224/0 (socket says 172.21.15.105:53224) 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e2ffd640 1 -- 172.21.15.105:0/2693929695 learned_addr learned my addr 172.21.15.105:0/2693929695 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e2ffd640 1 -- 172.21.15.105:0/2693929695 >> v2:172.21.15.144:3300/0 conn(0x7fb3e415bea0 msgr2=0x7fb3e40b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/2693929695 >> v2:172.21.15.144:3300/0 conn(0x7fb3e415bea0 0x7fb3e40b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e2ffd640 1 -- 172.21.15.105:0/2693929695 >> v1:172.21.15.144:6789/0 conn(0x7fb3e40b5a60 legacy=0x7fb3e409baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.781+0000 7fb3e2ffd640 1 -- 172.21.15.105:0/2693929695 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e415ecd0 con 0x7fb3e415df30 2024-03-08T08:27:40.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.782+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/2693929695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415df30 0x7fb3e40b5520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009870 tx=0x7fb3d402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87059304e2063e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.782+0000 7fb3e27fc640 1 -- 172.21.15.105:0/2693929695 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4034070 con 0x7fb3e415df30 2024-03-08T08:27:40.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.782+0000 7fb3e27fc640 1 -- 172.21.15.105:0/2693929695 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4030840 con 0x7fb3e415df30 2024-03-08T08:27:40.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.782+0000 7fb3e27fc640 1 -- 172.21.15.105:0/2693929695 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4030c10 con 0x7fb3e415df30 2024-03-08T08:27:40.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.783+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/2693929695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415df30 msgr2=0x7fb3e40b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.783+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/2693929695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415df30 0x7fb3e40b5520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009870 tx=0x7fb3d402b980 comp rx=0 tx=0).stop 2024-03-08T08:27:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.783+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/2693929695 shutdown_connections 2024-03-08T08:27:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.783+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/2693929695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415df30 0x7fb3e40b5520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.783+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/2693929695 >> v2:172.21.15.144:3300/0 conn(0x7fb3e415bea0 0x7fb3e40b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.783+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/2693929695 >> 172.21.15.105:0/2693929695 conn(0x7fb3e409aa00 msgr2=0x7fb3e409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.783+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/2693929695 shutdown_connections 2024-03-08T08:27:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.783+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/2693929695 wait complete. 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.784+0000 7fb3e9dc9640 1 Processor -- start 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.784+0000 7fb3e9dc9640 1 -- start start 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e9dc9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40b5a60 0x7fb3e41506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e9dc9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415bea0 0x7fb3e4152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e9dc9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e415df30 0x7fb3e4153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e9dc9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e415d290 con 0x7fb3e415df30 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e9dc9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e415f290 con 0x7fb3e40b5a60 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e9dc9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e415e3b0 con 0x7fb3e415bea0 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e415df30 0x7fb3e4153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e415df30 0x7fb3e4153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58820/0 (socket says 172.21.15.105:58820) 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 -- 172.21.15.105:0/1776318509 learned_addr learned my addr 172.21.15.105:0/1776318509 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e37fe640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40b5a60 0x7fb3e41506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415bea0 0x7fb3e4152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 -- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415bea0 msgr2=0x7fb3e4152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415bea0 0x7fb3e4152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 -- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40b5a60 msgr2=0x7fb3e41506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40b5a60 0x7fb3e41506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 -- 172.21.15.105:0/1776318509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e409bc40 con 0x7fb3e415df30 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e37fe640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40b5a60 0x7fb3e41506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e2ffd640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415bea0 0x7fb3e4152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e3fff640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e415df30 0x7fb3e4153000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00c730 tx=0x7fb3dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3cbfff640 1 -- 172.21.15.105:0/1776318509 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc017070 con 0x7fb3e415df30 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3cbfff640 1 -- 172.21.15.105:0/1776318509 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc00ce40 con 0x7fb3e415df30 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3cbfff640 1 -- 172.21.15.105:0/1776318509 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00a840 con 0x7fb3e415df30 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e409bf20 con 0x7fb3e415df30 2024-03-08T08:27:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.785+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e409c4e0 con 0x7fb3e415df30 2024-03-08T08:27:40.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.786+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4075120 con 0x7fb3e415df30 2024-03-08T08:27:40.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.787+0000 7fb3cbfff640 1 -- 172.21.15.105:0/1776318509 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3dc01a070 con 0x7fb3e415df30 2024-03-08T08:27:40.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.787+0000 7fb3cbfff640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb3d006b4d0 0x7fb3d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:40.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.787+0000 7fb3e37fe640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb3d006b4d0 0x7fb3d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:40.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.787+0000 7fb3cbfff640 1 -- 172.21.15.105:0/1776318509 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3dc0896d0 con 0x7fb3e415df30 2024-03-08T08:27:40.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.788+0000 7fb3e37fe640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb3d006b4d0 0x7fb3d006d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009af0 tx=0x7fb3d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.792+0000 7fb3cbfff640 1 -- 172.21.15.105:0/1776318509 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3dc057f00 con 0x7fb3e415df30 2024-03-08T08:27:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:40.999+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb3e4159480 con 0x7fb3e415df30 2024-03-08T08:27:41.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.669+0000 7fb3cbfff640 1 -- 172.21.15.105:0/1776318509 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3dc05bfa0 con 0x7fb3e415df30 2024-03-08T08:27:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb3d006b4d0 msgr2=0x7fb3d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb3d006b4d0 0x7fb3d006d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009af0 tx=0x7fb3d8008040 comp rx=0 tx=0).stop 2024-03-08T08:27:41.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e415df30 msgr2=0x7fb3e4153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e415df30 0x7fb3e4153000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00c730 tx=0x7fb3dc00cbf0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 shutdown_connections 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb3d006b4d0 0x7fb3d006d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3e415df30 0x7fb3e4153000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e415bea0 0x7fb3e4152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 --2- 172.21.15.105:0/1776318509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40b5a60 0x7fb3e41506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 >> 172.21.15.105:0/1776318509 conn(0x7fb3e409aa00 msgr2=0x7fb3e40b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 shutdown_connections 2024-03-08T08:27:41.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.672+0000 7fb3e9dc9640 1 -- 172.21.15.105:0/1776318509 wait complete. 2024-03-08T08:27:41.686 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-08T08:27:41.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:27:41.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.789+0000 7f59f4763640 1 Processor -- start 2024-03-08T08:27:41.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.789+0000 7f59f4763640 1 -- start start 2024-03-08T08:27:41.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f4763640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59ec0ceb20 0x7f59ec1621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:41.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f4763640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec1627c0 con 0x7f59ec15df20 2024-03-08T08:27:41.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f4763640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec06f070 con 0x7f59ec0aea80 2024-03-08T08:27:41.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f4763640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec06ee20 con 0x7f59ec0ceb20 2024-03-08T08:27:41.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f2d00640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f59ec0ceb20 0x7f59ec1621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:41.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f24ff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f59ec15df20 0x7f59ec0ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57926/0 (socket says 172.21.15.105:57926) 2024-03-08T08:27:41.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f24ff640 1 -- 172.21.15.105:0/3723310928 learned_addr learned my addr 172.21.15.105:0/3723310928 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:41.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f2d00640 1 -- 172.21.15.105:0/3723310928 >> v1:172.21.15.144:6789/0 conn(0x7f59ec0aea80 legacy=0x7f59ec161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:41.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f2d00640 1 -- 172.21.15.105:0/3723310928 >> v1:172.21.15.105:6789/0 conn(0x7f59ec15df20 legacy=0x7f59ec0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:41.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f14fd640 1 -- 172.21.15.105:0/3723310928 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391232782 0 0) 0x7f59ec1627c0 con 0x7f59ec15df20 2024-03-08T08:27:41.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.790+0000 7f59f2d00640 1 -- 172.21.15.105:0/3723310928 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec06f350 con 0x7f59ec0ceb20 2024-03-08T08:27:41.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f2d00640 1 --2- 172.21.15.105:0/3723310928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec0ceb20 0x7f59ec1621a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009d90 tx=0x7f59e402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2c284411607af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:41.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f14fd640 1 -- 172.21.15.105:0/3723310928 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4039070 con 0x7f59ec0ceb20 2024-03-08T08:27:41.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f14fd640 1 -- 172.21.15.105:0/3723310928 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e402cc60 con 0x7f59ec0ceb20 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f14fd640 1 -- 172.21.15.105:0/3723310928 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4034660 con 0x7f59ec0ceb20 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f4763640 1 -- 172.21.15.105:0/3723310928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec0ceb20 msgr2=0x7f59ec1621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f4763640 1 --2- 172.21.15.105:0/3723310928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec0ceb20 0x7f59ec1621a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009d90 tx=0x7f59e402beb0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f4763640 1 -- 172.21.15.105:0/3723310928 shutdown_connections 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f4763640 1 --2- 172.21.15.105:0/3723310928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec0ceb20 0x7f59ec1621a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f4763640 1 -- 172.21.15.105:0/3723310928 >> 172.21.15.105:0/3723310928 conn(0x7f59ec09aa00 msgr2=0x7f59ec09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f4763640 1 -- 172.21.15.105:0/3723310928 shutdown_connections 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.791+0000 7f59f4763640 1 -- 172.21.15.105:0/3723310928 wait complete. 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f4763640 1 Processor -- start 2024-03-08T08:27:41.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f4763640 1 -- start start 2024-03-08T08:27:41.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f4763640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec0aea80 0x7f59ec0b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:41.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f4763640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec0ceb20 0x7f59ec154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:41.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f4763640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec15df20 0x7f59ec154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:41.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f4763640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec168f40 con 0x7f59ec15df20 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f4763640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec168dc0 con 0x7f59ec0aea80 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f4763640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec1690c0 con 0x7f59ec0ceb20 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f2d00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec15df20 0x7f59ec154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f2d00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec15df20 0x7f59ec154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58836/0 (socket says 172.21.15.105:58836) 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.793+0000 7f59f2d00640 1 -- 172.21.15.105:0/251067043 learned_addr learned my addr 172.21.15.105:0/251067043 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f1cfe640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec0ceb20 0x7f59ec154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f24ff640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec0aea80 0x7f59ec0b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f2d00640 1 -- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec0ceb20 msgr2=0x7f59ec154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f2d00640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec0ceb20 0x7f59ec154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f2d00640 1 -- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec0aea80 msgr2=0x7f59ec0b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f2d00640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec0aea80 0x7f59ec0b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f2d00640 1 -- 172.21.15.105:0/251067043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0b42f0 con 0x7f59ec15df20 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f1cfe640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec0ceb20 0x7f59ec154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:41.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f24ff640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec0aea80 0x7f59ec0b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f2d00640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec15df20 0x7f59ec154bc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009d90 tx=0x7f59e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59e2ffd640 1 -- 172.21.15.105:0/251067043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4044020 con 0x7f59ec15df20 2024-03-08T08:27:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59e2ffd640 1 -- 172.21.15.105:0/251067043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e403f470 con 0x7f59ec15df20 2024-03-08T08:27:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec1f78a0 con 0x7f59ec15df20 2024-03-08T08:27:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59e2ffd640 1 -- 172.21.15.105:0/251067043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e4039040 con 0x7f59ec15df20 2024-03-08T08:27:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.794+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec1f7b60 con 0x7f59ec15df20 2024-03-08T08:27:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.795+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec075120 con 0x7f59ec15df20 2024-03-08T08:27:41.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.796+0000 7f59e2ffd640 1 -- 172.21.15.105:0/251067043 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59e4047020 con 0x7f59ec15df20 2024-03-08T08:27:41.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.796+0000 7f59e2ffd640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f59d006b500 0x7f59d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:41.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.796+0000 7f59e2ffd640 1 -- 172.21.15.105:0/251067043 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59e40acc00 con 0x7f59ec15df20 2024-03-08T08:27:41.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.796+0000 7f59f24ff640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f59d006b500 0x7f59d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:41.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.796+0000 7f59f24ff640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f59d006b500 0x7f59d006d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59e8000f60 tx=0x7f59e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:41.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:41.800+0000 7f59e2ffd640 1 -- 172.21.15.105:0/251067043 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59e40b3080 con 0x7f59ec15df20 2024-03-08T08:27:42.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.006+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59ec0cf1d0 con 0x7f59ec15df20 2024-03-08T08:27:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.007+0000 7f59e2ffd640 1 -- 172.21.15.105:0/251067043 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4944 (secure 0 0 0) 0x7f59e404e070 con 0x7f59ec15df20 2024-03-08T08:27:42.009 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:42.009 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:27:38.998370+0000","modified":"2024-03-08T08:27:41.001533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T08:27:42.009 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-08T08:27:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f59d006b500 msgr2=0x7f59d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f59d006b500 0x7f59d006d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59e8000f60 tx=0x7f59e8009040 comp rx=0 tx=0).stop 2024-03-08T08:27:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec15df20 msgr2=0x7f59ec154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec15df20 0x7f59ec154bc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009d90 tx=0x7f59e4004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 shutdown_connections 2024-03-08T08:27:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f59d006b500 0x7f59d006d9c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59ec15df20 0x7f59ec154bc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec0ceb20 0x7f59ec154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 --2- 172.21.15.105:0/251067043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec0aea80 0x7f59ec0b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 >> 172.21.15.105:0/251067043 conn(0x7f59ec09aa00 msgr2=0x7f59ec156190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 shutdown_connections 2024-03-08T08:27:42.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.009+0000 7f59f4763640 1 -- 172.21.15.105:0/251067043 wait complete. 2024-03-08T08:27:42.021 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:42.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.118+0000 7f6292c91640 1 Processor -- start 2024-03-08T08:27:42.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.119+0000 7f6292c91640 1 -- start start 2024-03-08T08:27:42.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.119+0000 7f6292c91640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c161520 0x7f628c0b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.119+0000 7f6292c91640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f628c0ce790 0x7f628c0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.119+0000 7f6292c91640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0b29e0 con 0x7f628c0ce330 2024-03-08T08:27:42.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.119+0000 7f6292c91640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c0d1930 con 0x7f628c0ce790 2024-03-08T08:27:42.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.119+0000 7f6292c91640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c06c770 con 0x7f628c161520 2024-03-08T08:27:42.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f6290a2d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c161520 0x7f628c0b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f629122e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f628c0ce790 0x7f628c0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f6290a2d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c161520 0x7f628c0b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:54820/0 (socket says 172.21.15.105:54820) 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f629122e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f628c0ce790 0x7f628c0b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59526/0 (socket says 172.21.15.105:59526) 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f6290a2d640 1 -- 172.21.15.105:0/673122858 learned_addr learned my addr 172.21.15.105:0/673122858 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f628bfff640 1 --1- 172.21.15.105:0/673122858 >> v1:172.21.15.144:6789/0 conn(0x7f628c0ce330 0x7f628c0b14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:53244/0 (socket says 172.21.15.105:53244) 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f629122e640 1 -- 172.21.15.105:0/673122858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c161520 msgr2=0x7f628c0b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f629122e640 1 --2- 172.21.15.105:0/673122858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c161520 0x7f628c0b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f629122e640 1 -- 172.21.15.105:0/673122858 >> v1:172.21.15.144:6789/0 conn(0x7f628c0ce330 legacy=0x7f628c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f629122e640 1 -- 172.21.15.105:0/673122858 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0d1650 con 0x7f628c0ce790 2024-03-08T08:27:42.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.120+0000 7f6290a2d640 1 --2- 172.21.15.105:0/673122858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c161520 0x7f628c0b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f629122e640 1 --2- 172.21.15.105:0/673122858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c0ce790 0x7f628c0b1c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6284009d90 tx=0x7f628402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e00720d267fa9e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f628b7fe640 1 -- 172.21.15.105:0/673122858 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284039070 con 0x7f628c0ce790 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f628b7fe640 1 -- 172.21.15.105:0/673122858 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628402cc50 con 0x7f628c0ce790 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f628b7fe640 1 -- 172.21.15.105:0/673122858 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284034440 con 0x7f628c0ce790 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f6292c91640 1 -- 172.21.15.105:0/673122858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c0ce790 msgr2=0x7f628c0b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f6292c91640 1 --2- 172.21.15.105:0/673122858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c0ce790 0x7f628c0b1c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6284009d90 tx=0x7f628402bea0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f6292c91640 1 -- 172.21.15.105:0/673122858 shutdown_connections 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f6292c91640 1 --2- 172.21.15.105:0/673122858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c0ce790 0x7f628c0b1c00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f6292c91640 1 --2- 172.21.15.105:0/673122858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c161520 0x7f628c0b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f6292c91640 1 -- 172.21.15.105:0/673122858 >> 172.21.15.105:0/673122858 conn(0x7f628c0b36a0 msgr2=0x7f628c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f6292c91640 1 -- 172.21.15.105:0/673122858 shutdown_connections 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.121+0000 7f6292c91640 1 -- 172.21.15.105:0/673122858 wait complete. 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.123+0000 7f6292c91640 1 Processor -- start 2024-03-08T08:27:42.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.123+0000 7f6292c91640 1 -- start start 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.123+0000 7f6292c91640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c0ce330 0x7f628c1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.123+0000 7f6292c91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c0ce790 0x7f628c0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.123+0000 7f6292c91640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c161520 0x7f628c0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.123+0000 7f6292c91640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c161900 con 0x7f628c0ce790 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.123+0000 7f6292c91640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c06ca40 con 0x7f628c161520 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.123+0000 7f6292c91640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b32d0 con 0x7f628c0ce330 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c0ce790 0x7f628c0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c0ce790 0x7f628c0b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58838/0 (socket says 172.21.15.105:58838) 2024-03-08T08:27:42.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 -- 172.21.15.105:0/1469465206 learned_addr learned my addr 172.21.15.105:0/1469465206 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f6290a2d640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c0ce330 0x7f628c1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f629122e640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c161520 0x7f628c0af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 -- 172.21.15.105:0/1469465206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c0ce330 msgr2=0x7f628c1591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c0ce330 0x7f628c1591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 -- 172.21.15.105:0/1469465206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c161520 msgr2=0x7f628c0af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c161520 0x7f628c0af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 -- 172.21.15.105:0/1469465206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0c8fb0 con 0x7f628c0ce790 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f628bfff640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c0ce790 0x7f628c0b08d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f627800c400 tx=0x7f627800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f62897fa640 1 -- 172.21.15.105:0/1469465206 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780074e0 con 0x7f628c0ce790 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f62897fa640 1 -- 172.21.15.105:0/1469465206 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6278007680 con 0x7f628c0ce790 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0c9290 con 0x7f628c0ce790 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f62897fa640 1 -- 172.21.15.105:0/1469465206 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278004a00 con 0x7f628c0ce790 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.124+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c15a9d0 con 0x7f628c0ce790 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.125+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c075120 con 0x7f628c0ce790 2024-03-08T08:27:42.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.125+0000 7f62897fa640 1 -- 172.21.15.105:0/1469465206 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f627801a020 con 0x7f628c0ce790 2024-03-08T08:27:42.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.126+0000 7f62897fa640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.126+0000 7f62897fa640 1 -- 172.21.15.105:0/1469465206 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62780898b0 con 0x7f628c0ce790 2024-03-08T08:27:42.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.126+0000 7f6290a2d640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.126+0000 7f6290a2d640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f627006b4d0 0x7f627006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6280006ca0 tx=0x7f6280009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.130+0000 7f62897fa640 1 -- 172.21.15.105:0/1469465206 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62780580e0 con 0x7f628c0ce790 2024-03-08T08:27:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.299+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f628c15cbd0 con 0x7f628c0ce790 2024-03-08T08:27:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.301+0000 7f62897fa640 1 -- 172.21.15.105:0/1469465206 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f627805c180 con 0x7f628c0ce790 2024-03-08T08:27:42.303 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:42.303 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.302+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f627006b4d0 msgr2=0x7f627006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.302+0000 7f6292c91640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f627006b4d0 0x7f627006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6280006ca0 tx=0x7f6280009240 comp rx=0 tx=0).stop 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.302+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c0ce790 msgr2=0x7f628c0b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.302+0000 7f6292c91640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c0ce790 0x7f628c0b08d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f627800c400 tx=0x7f627800c8c0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.303+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 shutdown_connections 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.303+0000 7f6292c91640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.303+0000 7f6292c91640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628c161520 0x7f628c0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.303+0000 7f6292c91640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628c0ce790 0x7f628c0b08d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.303+0000 7f6292c91640 1 --2- 172.21.15.105:0/1469465206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f628c0ce330 0x7f628c1591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.303+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 >> 172.21.15.105:0/1469465206 conn(0x7f628c0b36a0 msgr2=0x7f628c1607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:42.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.303+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 shutdown_connections 2024-03-08T08:27:42.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.303+0000 7f6292c91640 1 -- 172.21.15.105:0/1469465206 wait complete. 2024-03-08T08:27:42.315 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-08T08:27:42.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:27:42.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.413+0000 7fe2e646b640 1 Processor -- start 2024-03-08T08:27:42.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.413+0000 7fe2e646b640 1 -- start start 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2e646b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 0x7fe2e015f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2e646b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e00b2c20 con 0x7fe2e00b2740 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2e646b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e00af4c0 con 0x7fe2e00cfdb0 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2e646b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00af7a0 con 0x7fe2e00d06e0 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2e4a08640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe2e00b2740 0x7fe2e0165b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57928/0 (socket says 172.21.15.105:57928) 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2e4a08640 1 -- 172.21.15.105:0/3437325867 learned_addr learned my addr 172.21.15.105:0/3437325867 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2df7fe640 1 --2- 172.21.15.105:0/3437325867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 0x7fe2e015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2deffd640 1 -- 172.21.15.105:0/3437325867 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314017969 0 0) 0x7fe2e00b2c20 con 0x7fe2e00b2740 2024-03-08T08:27:42.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2deffd640 1 -- 172.21.15.105:0/3437325867 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c8003520 con 0x7fe2e00b2740 2024-03-08T08:27:42.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2deffd640 1 -- 172.21.15.105:0/3437325867 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3105301529 0 0) 0x7fe2c8003520 con 0x7fe2e00b2740 2024-03-08T08:27:42.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2df7fe640 1 -- 172.21.15.105:0/3437325867 >> v1:172.21.15.144:6789/0 conn(0x7fe2e00cfdb0 legacy=0x7fe2e015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2df7fe640 1 -- 172.21.15.105:0/3437325867 >> v1:172.21.15.105:6789/0 conn(0x7fe2e00b2740 legacy=0x7fe2e0165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.414+0000 7fe2df7fe640 1 -- 172.21.15.105:0/3437325867 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00cc490 con 0x7fe2e00d06e0 2024-03-08T08:27:42.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.415+0000 7fe2df7fe640 1 --2- 172.21.15.105:0/3437325867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 0x7fe2e015f580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00097d0 tx=0x7fe2d002b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104243671ba5831a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.415+0000 7fe2deffd640 1 -- 172.21.15.105:0/3437325867 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0037070 con 0x7fe2e00d06e0 2024-03-08T08:27:42.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.415+0000 7fe2deffd640 1 -- 172.21.15.105:0/3437325867 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d0030510 con 0x7fe2e00d06e0 2024-03-08T08:27:42.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.415+0000 7fe2deffd640 1 -- 172.21.15.105:0/3437325867 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d80019f0 con 0x7fe2e00d06e0 2024-03-08T08:27:42.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.416+0000 7fe2e646b640 1 -- 172.21.15.105:0/3437325867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 msgr2=0x7fe2e015f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.416+0000 7fe2e646b640 1 --2- 172.21.15.105:0/3437325867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 0x7fe2e015f580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2d00097d0 tx=0x7fe2d002b5d0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.416+0000 7fe2e646b640 1 -- 172.21.15.105:0/3437325867 shutdown_connections 2024-03-08T08:27:42.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.416+0000 7fe2e646b640 1 --2- 172.21.15.105:0/3437325867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 0x7fe2e015f580 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.416+0000 7fe2e646b640 1 -- 172.21.15.105:0/3437325867 >> 172.21.15.105:0/3437325867 conn(0x7fe2e00b36a0 msgr2=0x7fe2e00b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:42.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.416+0000 7fe2e646b640 1 -- 172.21.15.105:0/3437325867 shutdown_connections 2024-03-08T08:27:42.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.416+0000 7fe2e646b640 1 -- 172.21.15.105:0/3437325867 wait complete. 2024-03-08T08:27:42.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.417+0000 7fe2e646b640 1 Processor -- start 2024-03-08T08:27:42.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.417+0000 7fe2e646b640 1 -- start start 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2e646b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e00b2740 0x7fe2e014cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2e646b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e00cfdb0 0x7fe2e00b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2e646b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 0x7fe2e0157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2e646b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00b1f90 con 0x7fe2e00b2740 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2e646b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00d0460 con 0x7fe2e00cfdb0 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2e646b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00b2110 con 0x7fe2e00d06e0 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e00b2740 0x7fe2e014cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e00b2740 0x7fe2e014cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58846/0 (socket says 172.21.15.105:58846) 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 -- 172.21.15.105:0/2007080480 learned_addr learned my addr 172.21.15.105:0/2007080480 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:42.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 -- 172.21.15.105:0/2007080480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 msgr2=0x7fe2e0157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 0x7fe2e0157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 -- 172.21.15.105:0/2007080480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e00cfdb0 msgr2=0x7fe2e00b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e00cfdb0 0x7fe2e00b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 -- 172.21.15.105:0/2007080480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00b4790 con 0x7fe2e00b2740 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dffff640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e00b2740 0x7fe2e014cf00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c780 tx=0x7fe2d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dcff9640 1 -- 172.21.15.105:0/2007080480 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4017070 con 0x7fe2e00b2740 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dcff9640 1 -- 172.21.15.105:0/2007080480 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d4005290 con 0x7fe2e00b2740 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00b4a70 con 0x7fe2e00b2740 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2dcff9640 1 -- 172.21.15.105:0/2007080480 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400a880 con 0x7fe2e00b2740 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.418+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e00b5030 con 0x7fe2e00b2740 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.419+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e0067ec0 con 0x7fe2e00b2740 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.420+0000 7fe2dcff9640 1 -- 172.21.15.105:0/2007080480 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2d401a070 con 0x7fe2e00b2740 2024-03-08T08:27:42.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.420+0000 7fe2dcff9640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.420+0000 7fe2df7fe640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.420+0000 7fe2dcff9640 1 -- 172.21.15.105:0/2007080480 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2d4089730 con 0x7fe2e00b2740 2024-03-08T08:27:42.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.421+0000 7fe2df7fe640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0034000 tx=0x7fe2d002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.425+0000 7fe2dcff9640 1 -- 172.21.15.105:0/2007080480 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2e0067ec0 con 0x7fe2e00b2740 2024-03-08T08:27:42.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.592+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2e00ce880 con 0x7fe2cc06b4d0 2024-03-08T08:27:42.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.598+0000 7fe2dcff9640 1 -- 172.21.15.105:0/2007080480 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237151 (secure 0 0 0) 0x7fe2e00ce880 con 0x7fe2cc06b4d0 2024-03-08T08:27:42.603 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:42.604 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe2cc06b4d0 msgr2=0x7fe2cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0034000 tx=0x7fe2d002f000 comp rx=0 tx=0).stop 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e00b2740 msgr2=0x7fe2e014cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e00b2740 0x7fe2e014cf00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c780 tx=0x7fe2d400cc40 comp rx=0 tx=0).stop 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 shutdown_connections 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00d06e0 0x7fe2e0157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e00cfdb0 0x7fe2e00b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 --2- 172.21.15.105:0/2007080480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e00b2740 0x7fe2e014cf00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 >> 172.21.15.105:0/2007080480 conn(0x7fe2e00b36a0 msgr2=0x7fe2e006e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 shutdown_connections 2024-03-08T08:27:42.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.601+0000 7fe2e646b640 1 -- 172.21.15.105:0/2007080480 wait complete. 2024-03-08T08:27:42.614 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-08T08:27:41.095986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219728,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754992,"statfs":{"total":773094113280,"available":772869111808,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000707"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-08T08:27:31.283233+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:27:31.283233+0000","last_peered":"2024-03-08T08:27:31.283233+0000","last_clean":"2024-03-08T08:27:31.283233+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:27:31.283233+0000","last_undegraded":"2024-03-08T08:27:31.283233+0000","last_fullsized":"2024-03-08T08:27:31.283233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28376,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343464,"statfs":{"total":96636764160,"available":96607707136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:27:42.618 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:27:42.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.752+0000 7fe1ce93e640 1 Processor -- start 2024-03-08T08:27:42.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.752+0000 7fe1ce93e640 1 -- start start 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1ce93e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe1c80b2b90 0x7fe1c80b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1ce93e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80b1af0 con 0x7fe1c815dd80 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1ce93e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80b1dd0 con 0x7fe1c80b1550 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1ce93e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c80b2700 con 0x7fe1c80b2b90 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1c7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe1c815dd80 0x7fe1c81623c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57934/0 (socket says 172.21.15.105:57934) 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1c7fff640 1 -- 172.21.15.105:0/2298110336 learned_addr learned my addr 172.21.15.105:0/2298110336 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1ccedb640 1 --2- 172.21.15.105:0/2298110336 >> v2:172.21.15.144:3300/0 conn(0x7fe1c80b2b90 0x7fe1c80b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868006414 0 0) 0x7fe1c80b1af0 con 0x7fe1c815dd80 2024-03-08T08:27:42.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.753+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b0003520 con 0x7fe1c815dd80 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2371639673 0 0) 0x7fe1b0003520 con 0x7fe1c815dd80 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 >> v2:172.21.15.144:3300/0 conn(0x7fe1c80b2b90 msgr2=0x7fe1c80b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 --2- 172.21.15.105:0/2298110336 >> v2:172.21.15.144:3300/0 conn(0x7fe1c80b2b90 0x7fe1c80b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 >> v1:172.21.15.144:6789/0 conn(0x7fe1c80b1550 legacy=0x7fe1c80aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80b20b0 con 0x7fe1c815dd80 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fe1b8001670 con 0x7fe1c815dd80 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 >> v1:172.21.15.105:6789/0 conn(0x7fe1c815dd80 legacy=0x7fe1c81623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b00039b0 0x7fe1b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b0004800 0x7fe1b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b00051c0 0x7fe1b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80b2700 con 0x7fe1b0004800 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b000c060 con 0x7fe1b00051c0 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b000c260 con 0x7fe1b00039b0 2024-03-08T08:27:42.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c77fe640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b0004800 0x7fe1b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1ccedb640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b00051c0 0x7fe1b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c7fff640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b00039b0 0x7fe1b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c77fe640 1 -- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b00039b0 msgr2=0x7fe1b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c77fe640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b00039b0 0x7fe1b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c77fe640 1 -- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b00051c0 msgr2=0x7fe1b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c77fe640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b00051c0 0x7fe1b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1ccedb640 1 -- 172.21.15.105:0/2298110336 reap_dead start 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c77fe640 1 -- 172.21.15.105:0/2298110336 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b0003520 con 0x7fe1b0004800 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.754+0000 7fe1c7fff640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1b00039b0 0x7fe1b0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.755+0000 7fe1c77fe640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b0004800 0x7fe1b0004c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc001750 tx=0x7fe1b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdeb00a34744fbd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.755+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc001370 con 0x7fe1b0004800 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.755+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc0052d0 con 0x7fe1b0004800 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.755+0000 7fe1c6ffd640 1 -- 172.21.15.105:0/2298110336 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc003830 con 0x7fe1b0004800 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.755+0000 7fe1ce93e640 1 -- 172.21.15.105:0/2298110336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b0004800 msgr2=0x7fe1b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.755+0000 7fe1ce93e640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b0004800 0x7fe1b0004c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc001750 tx=0x7fe1b0005830 comp rx=0 tx=0).stop 2024-03-08T08:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.755+0000 7fe1ce93e640 1 -- 172.21.15.105:0/2298110336 shutdown_connections 2024-03-08T08:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.755+0000 7fe1ce93e640 1 --2- 172.21.15.105:0/2298110336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1b0004800 0x7fe1b0004c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.756+0000 7fe1ce93e640 1 -- 172.21.15.105:0/2298110336 >> 172.21.15.105:0/2298110336 conn(0x7fe1c80b37a0 msgr2=0x7fe1c80b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.756+0000 7fe1ce93e640 1 -- 172.21.15.105:0/2298110336 shutdown_connections 2024-03-08T08:27:42.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.756+0000 7fe1ce93e640 1 -- 172.21.15.105:0/2298110336 wait complete. 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.757+0000 7fe1ce93e640 1 Processor -- start 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.757+0000 7fe1ce93e640 1 -- start start 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.757+0000 7fe1ce93e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b0004800 0x7fe1c8160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.757+0000 7fe1ce93e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1c81f3a80 0x7fe1c81f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.757+0000 7fe1ce93e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c8158a30 0x7fe1c8158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ce93e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c815e160 con 0x7fe1c8158a30 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ce93e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c81623c0 con 0x7fe1b0004800 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ce93e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c81f4c40 con 0x7fe1c81f3a80 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c8158a30 0x7fe1c8158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c8158a30 0x7fe1c8158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58852/0 (socket says 172.21.15.105:58852) 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 -- 172.21.15.105:0/3401139110 learned_addr learned my addr 172.21.15.105:0/3401139110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:42.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1c7fff640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b0004800 0x7fe1c8160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1c77fe640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1c81f3a80 0x7fe1c81f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 -- 172.21.15.105:0/3401139110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1c81f3a80 msgr2=0x7fe1c81f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1c81f3a80 0x7fe1c81f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 -- 172.21.15.105:0/3401139110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b0004800 msgr2=0x7fe1c8160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b0004800 0x7fe1c8160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 -- 172.21.15.105:0/3401139110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c81f45b0 con 0x7fe1c8158a30 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ccedb640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c8158a30 0x7fe1c8158eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80b7d60 tx=0x7fe1c000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1c4ff9640 1 -- 172.21.15.105:0/3401139110 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000fce0 con 0x7fe1c8158a30 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1c4ff9640 1 -- 172.21.15.105:0/3401139110 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000fe80 con 0x7fe1c8158a30 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c81f4890 con 0x7fe1c8158a30 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1c4ff9640 1 -- 172.21.15.105:0/3401139110 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0015670 con 0x7fe1c8158a30 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.758+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c815fdc0 con 0x7fe1c8158a30 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.759+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c80cc0d0 con 0x7fe1c8158a30 2024-03-08T08:27:42.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.759+0000 7fe1c4ff9640 1 -- 172.21.15.105:0/3401139110 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1c0015810 con 0x7fe1c8158a30 2024-03-08T08:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.760+0000 7fe1c4ff9640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1b406b4d0 0x7fe1b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.760+0000 7fe1c4ff9640 1 -- 172.21.15.105:0/3401139110 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1c008b0a0 con 0x7fe1c8158a30 2024-03-08T08:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.760+0000 7fe1c7fff640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1b406b4d0 0x7fe1b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.760+0000 7fe1c7fff640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1b406b4d0 0x7fe1b406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8001cd0 tx=0x7fe1b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:42.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.764+0000 7fe1c4ff9640 1 -- 172.21.15.105:0/3401139110 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1c0019030 con 0x7fe1c8158a30 2024-03-08T08:27:42.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.931+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1c81f4a60 con 0x7fe1b406b4d0 2024-03-08T08:27:42.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.937+0000 7fe1c4ff9640 1 -- 172.21.15.105:0/3401139110 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237151 (secure 0 0 0) 0x7fe1b80107e0 con 0x7fe1b406b4d0 2024-03-08T08:27:42.941 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:42.942 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1b406b4d0 msgr2=0x7fe1b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1b406b4d0 0x7fe1b406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8001cd0 tx=0x7fe1b8009040 comp rx=0 tx=0).stop 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c8158a30 msgr2=0x7fe1c8158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c8158a30 0x7fe1c8158eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe1c80b7d60 tx=0x7fe1c000f1b0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 shutdown_connections 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1b406b4d0 0x7fe1b406d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1c8158a30 0x7fe1c8158eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe1c81f3a80 0x7fe1c81f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 --2- 172.21.15.105:0/3401139110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1b0004800 0x7fe1c8160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 >> 172.21.15.105:0/3401139110 conn(0x7fe1c80b37a0 msgr2=0x7fe1c806d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:42.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 shutdown_connections 2024-03-08T08:27:42.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:42.940+0000 7fe1ce93e640 1 -- 172.21.15.105:0/3401139110 wait complete. 2024-03-08T08:27:42.952 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-08T08:27:41.095986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219728,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754992,"statfs":{"total":773094113280,"available":772869111808,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000707"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-08T08:27:31.283233+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:27:31.283233+0000","last_peered":"2024-03-08T08:27:31.283233+0000","last_clean":"2024-03-08T08:27:31.283233+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:27:31.283233+0000","last_undegraded":"2024-03-08T08:27:31.283233+0000","last_fullsized":"2024-03-08T08:27:31.283233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:38.637143+0000","last_change":"2024-03-08T08:27:38.637143+0000","last_active":"2024-03-08T08:27:38.637143+0000","last_peered":"2024-03-08T08:27:38.637143+0000","last_clean":"2024-03-08T08:27:38.637143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:38.637143+0000","last_undegraded":"2024-03-08T08:27:38.637143+0000","last_fullsized":"2024-03-08T08:27:38.637143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-08T08:27:37.632553+0000","last_change":"2024-03-08T08:27:37.632553+0000","last_active":"2024-03-08T08:27:37.632553+0000","last_peered":"2024-03-08T08:27:37.632553+0000","last_clean":"2024-03-08T08:27:37.632553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:27:37.632553+0000","last_undegraded":"2024-03-08T08:27:37.632553+0000","last_fullsized":"2024-03-08T08:27:37.632553+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28376,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343464,"statfs":{"total":96636764160,"available":96607707136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:27:42.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:27:43.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.090+0000 7f3fb5193640 1 Processor -- start 2024-03-08T08:27:43.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.090+0000 7f3fb5193640 1 -- start start 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fb5193640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3fb015dd80 0x7f3fb01623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fb5193640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fb00b1550 0x7f3fb00aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fb5193640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb0162900 con 0x7f3fb00b2b90 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fb5193640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb00b2700 con 0x7f3fb00b1550 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fb5193640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb00b1af0 con 0x7f3fb015dd80 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fae59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3fb00b1550 0x7f3fb00aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3faf59e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3fb00b2b90 0x7f3fb00b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57944/0 (socket says 172.21.15.105:57944) 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3faf59e640 1 -- 172.21.15.105:0/1785614267 learned_addr learned my addr 172.21.15.105:0/1785614267 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3faed9d640 1 --2- 172.21.15.105:0/1785614267 >> v2:172.21.15.144:3300/0 conn(0x7f3fb015dd80 0x7f3fb01623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fae59c640 1 -- 172.21.15.105:0/1785614267 >> v2:172.21.15.144:3300/0 conn(0x7f3fb015dd80 msgr2=0x7f3fb01623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:43.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fae59c640 1 --2- 172.21.15.105:0/1785614267 >> v2:172.21.15.144:3300/0 conn(0x7f3fb015dd80 0x7f3fb01623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fae59c640 1 -- 172.21.15.105:0/1785614267 >> v1:172.21.15.105:6789/0 conn(0x7f3fb00b2b90 legacy=0x7f3fb00b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fadd9b640 1 -- 172.21.15.105:0/1785614267 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621203824 0 0) 0x7f3fa8001490 con 0x7f3fb00b2b90 2024-03-08T08:27:43.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.091+0000 7f3fae59c640 1 -- 172.21.15.105:0/1785614267 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00b1dd0 con 0x7f3fb00b1550 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.092+0000 7f3fae59c640 1 --2- 172.21.15.105:0/1785614267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b1550 0x7f3fb00aea80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40097a0 tx=0x7f3fa402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1c6db3625e01a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.092+0000 7f3fadd9b640 1 -- 172.21.15.105:0/1785614267 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4035020 con 0x7f3fb00b1550 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.092+0000 7f3fadd9b640 1 -- 172.21.15.105:0/1785614267 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa4031830 con 0x7f3fb00b1550 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.092+0000 7f3fadd9b640 1 -- 172.21.15.105:0/1785614267 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4031c00 con 0x7f3fb00b1550 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.092+0000 7f3fb5193640 1 -- 172.21.15.105:0/1785614267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b1550 msgr2=0x7f3fb00aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.092+0000 7f3fb5193640 1 --2- 172.21.15.105:0/1785614267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b1550 0x7f3fb00aea80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40097a0 tx=0x7f3fa402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.093+0000 7f3fb5193640 1 -- 172.21.15.105:0/1785614267 shutdown_connections 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.093+0000 7f3fb5193640 1 --2- 172.21.15.105:0/1785614267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b1550 0x7f3fb00aea80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.093+0000 7f3fb5193640 1 --2- 172.21.15.105:0/1785614267 >> v2:172.21.15.144:3300/0 conn(0x7f3fb015dd80 0x7f3fb01623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.093+0000 7f3fb5193640 1 -- 172.21.15.105:0/1785614267 >> 172.21.15.105:0/1785614267 conn(0x7f3fb00b37a0 msgr2=0x7f3fb00b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:43.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.093+0000 7f3fb5193640 1 -- 172.21.15.105:0/1785614267 shutdown_connections 2024-03-08T08:27:43.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.093+0000 7f3fb5193640 1 -- 172.21.15.105:0/1785614267 wait complete. 2024-03-08T08:27:43.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.094+0000 7f3fb5193640 1 Processor -- start 2024-03-08T08:27:43.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.094+0000 7f3fb5193640 1 -- start start 2024-03-08T08:27:43.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fb5193640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fb00b1550 0x7f3fb00b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fb5193640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b2b90 0x7f3fb0152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fb5193640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb015dd80 0x7f3fb0157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fb5193640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb015e160 con 0x7f3fb00b2b90 2024-03-08T08:27:43.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fb5193640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00b2390 con 0x7f3fb015dd80 2024-03-08T08:27:43.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fb5193640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00b7710 con 0x7f3fb00b1550 2024-03-08T08:27:43.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b2b90 0x7f3fb0152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b2b90 0x7f3fb0152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58870/0 (socket says 172.21.15.105:58870) 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 -- 172.21.15.105:0/570730547 learned_addr learned my addr 172.21.15.105:0/570730547 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3faed9d640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fb00b1550 0x7f3fb00b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3faf59e640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb015dd80 0x7f3fb0157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 -- 172.21.15.105:0/570730547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fb00b1550 msgr2=0x7f3fb00b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fb00b1550 0x7f3fb00b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 -- 172.21.15.105:0/570730547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb015dd80 msgr2=0x7f3fb0157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb015dd80 0x7f3fb0157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 -- 172.21.15.105:0/570730547 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0152b80 con 0x7f3fb00b2b90 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fae59c640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b2b90 0x7f3fb0152640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4034040 tx=0x7f3fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3f977fe640 1 -- 172.21.15.105:0/570730547 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4040070 con 0x7f3fb00b2b90 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3f977fe640 1 -- 172.21.15.105:0/570730547 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa40399a0 con 0x7f3fb00b2b90 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb00cd980 con 0x7f3fb00b2b90 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.095+0000 7f3f977fe640 1 -- 172.21.15.105:0/570730547 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4035040 con 0x7f3fb00b2b90 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.096+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb00cdf40 con 0x7f3fb00b2b90 2024-03-08T08:27:43.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.096+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb015ed50 con 0x7f3fb00b2b90 2024-03-08T08:27:43.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.097+0000 7f3f977fe640 1 -- 172.21.15.105:0/570730547 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fa4038020 con 0x7f3fb00b2b90 2024-03-08T08:27:43.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.097+0000 7f3f977fe640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.097+0000 7f3faed9d640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.098+0000 7f3f977fe640 1 -- 172.21.15.105:0/570730547 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fa40abbc0 con 0x7f3fb00b2b90 2024-03-08T08:27:43.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.098+0000 7f3faed9d640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006fd0 tx=0x7f3fa0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.102+0000 7f3f977fe640 1 -- 172.21.15.105:0/570730547 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fb015ed50 con 0x7f3fb00b2b90 2024-03-08T08:27:43.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.271+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fb0075120 con 0x7f3f9c06b4d0 2024-03-08T08:27:43.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.277+0000 7f3f977fe640 1 -- 172.21.15.105:0/570730547 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251896 (secure 0 0 0) 0x7f3fb0075120 con 0x7f3f9c06b4d0 2024-03-08T08:27:43.282 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:43.283 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:27:43.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f9c06b4d0 msgr2=0x7f3f9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0006fd0 tx=0x7f3fa0008040 comp rx=0 tx=0).stop 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b2b90 msgr2=0x7f3fb0152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b2b90 0x7f3fb0152640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4034040 tx=0x7f3fa40046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 shutdown_connections 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb015dd80 0x7f3fb0157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb00b2b90 0x7f3fb0152640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 --2- 172.21.15.105:0/570730547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fb00b1550 0x7f3fb00b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 >> 172.21.15.105:0/570730547 conn(0x7f3fb00b37a0 msgr2=0x7f3fb00cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 shutdown_connections 2024-03-08T08:27:43.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.280+0000 7f3fb5193640 1 -- 172.21.15.105:0/570730547 wait complete. 2024-03-08T08:27:43.293 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-08T08:27:43.096319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218896,"kb_used_data":2728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755824,"statfs":{"total":773094113280,"available":772869963776,"internally_reserved":0,"allocated":2793472,"data_stored":1702628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.458739"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.675515+0000","last_change":"2024-03-08T08:27:39.669170+0000","last_active":"2024-03-08T08:27:39.675515+0000","last_peered":"2024-03-08T08:27:39.675515+0000","last_clean":"2024-03-08T08:27:39.675515+0000","last_became_active":"2024-03-08T08:27:39.668776+0000","last_became_peered":"2024-03-08T08:27:39.668776+0000","last_unstale":"2024-03-08T08:27:39.675515+0000","last_undegraded":"2024-03-08T08:27:39.675515+0000","last_fullsized":"2024-03-08T08:27:39.675515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665542+0000","last_change":"2024-03-08T08:27:39.669937+0000","last_active":"2024-03-08T08:27:40.665542+0000","last_peered":"2024-03-08T08:27:40.665542+0000","last_clean":"2024-03-08T08:27:40.665542+0000","last_became_active":"2024-03-08T08:27:39.669643+0000","last_became_peered":"2024-03-08T08:27:39.669643+0000","last_unstale":"2024-03-08T08:27:40.665542+0000","last_undegraded":"2024-03-08T08:27:40.665542+0000","last_fullsized":"2024-03-08T08:27:40.665542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.675785+0000","last_change":"2024-03-08T08:27:39.669698+0000","last_active":"2024-03-08T08:27:39.675785+0000","last_peered":"2024-03-08T08:27:39.675785+0000","last_clean":"2024-03-08T08:27:39.675785+0000","last_became_active":"2024-03-08T08:27:39.669412+0000","last_became_peered":"2024-03-08T08:27:39.669412+0000","last_unstale":"2024-03-08T08:27:39.675785+0000","last_undegraded":"2024-03-08T08:27:39.675785+0000","last_fullsized":"2024-03-08T08:27:39.675785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:16.844234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681784+0000","last_change":"2024-03-08T08:27:39.674216+0000","last_active":"2024-03-08T08:27:39.681784+0000","last_peered":"2024-03-08T08:27:39.681784+0000","last_clean":"2024-03-08T08:27:39.681784+0000","last_became_active":"2024-03-08T08:27:39.673664+0000","last_became_peered":"2024-03-08T08:27:39.673664+0000","last_unstale":"2024-03-08T08:27:39.681784+0000","last_undegraded":"2024-03-08T08:27:39.681784+0000","last_fullsized":"2024-03-08T08:27:39.681784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.675489+0000","last_change":"2024-03-08T08:27:39.671956+0000","last_active":"2024-03-08T08:27:39.675489+0000","last_peered":"2024-03-08T08:27:39.675489+0000","last_clean":"2024-03-08T08:27:39.675489+0000","last_became_active":"2024-03-08T08:27:39.671671+0000","last_became_peered":"2024-03-08T08:27:39.671671+0000","last_unstale":"2024-03-08T08:27:39.675489+0000","last_undegraded":"2024-03-08T08:27:39.675489+0000","last_fullsized":"2024-03-08T08:27:39.675489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684558+0000","last_change":"2024-03-08T08:27:39.681569+0000","last_active":"2024-03-08T08:27:39.684558+0000","last_peered":"2024-03-08T08:27:39.684558+0000","last_clean":"2024-03-08T08:27:39.684558+0000","last_became_active":"2024-03-08T08:27:39.681306+0000","last_became_peered":"2024-03-08T08:27:39.681306+0000","last_unstale":"2024-03-08T08:27:39.684558+0000","last_undegraded":"2024-03-08T08:27:39.684558+0000","last_fullsized":"2024-03-08T08:27:39.684558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677011+0000","last_change":"2024-03-08T08:27:39.669471+0000","last_active":"2024-03-08T08:27:39.677011+0000","last_peered":"2024-03-08T08:27:39.677011+0000","last_clean":"2024-03-08T08:27:39.677011+0000","last_became_active":"2024-03-08T08:27:39.669144+0000","last_became_peered":"2024-03-08T08:27:39.669144+0000","last_unstale":"2024-03-08T08:27:39.677011+0000","last_undegraded":"2024-03-08T08:27:39.677011+0000","last_fullsized":"2024-03-08T08:27:39.677011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684664+0000","last_change":"2024-03-08T08:27:39.680587+0000","last_active":"2024-03-08T08:27:39.684664+0000","last_peered":"2024-03-08T08:27:39.684664+0000","last_clean":"2024-03-08T08:27:39.684664+0000","last_became_active":"2024-03-08T08:27:39.680354+0000","last_became_peered":"2024-03-08T08:27:39.680354+0000","last_unstale":"2024-03-08T08:27:39.684664+0000","last_undegraded":"2024-03-08T08:27:39.684664+0000","last_fullsized":"2024-03-08T08:27:39.684664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685850+0000","last_change":"2024-03-08T08:27:39.682744+0000","last_active":"2024-03-08T08:27:39.685850+0000","last_peered":"2024-03-08T08:27:39.685850+0000","last_clean":"2024-03-08T08:27:39.685850+0000","last_became_active":"2024-03-08T08:27:39.682486+0000","last_became_peered":"2024-03-08T08:27:39.682486+0000","last_unstale":"2024-03-08T08:27:39.685850+0000","last_undegraded":"2024-03-08T08:27:39.685850+0000","last_fullsized":"2024-03-08T08:27:39.685850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665493+0000","last_change":"2024-03-08T08:27:39.681720+0000","last_active":"2024-03-08T08:27:40.665493+0000","last_peered":"2024-03-08T08:27:40.665493+0000","last_clean":"2024-03-08T08:27:40.665493+0000","last_became_active":"2024-03-08T08:27:39.681473+0000","last_became_peered":"2024-03-08T08:27:39.681473+0000","last_unstale":"2024-03-08T08:27:40.665493+0000","last_undegraded":"2024-03-08T08:27:40.665493+0000","last_fullsized":"2024-03-08T08:27:40.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.681092+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.680824+0000","last_became_peered":"2024-03-08T08:27:39.680824+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665505+0000","last_change":"2024-03-08T08:27:39.674803+0000","last_active":"2024-03-08T08:27:40.665505+0000","last_peered":"2024-03-08T08:27:40.665505+0000","last_clean":"2024-03-08T08:27:40.665505+0000","last_became_active":"2024-03-08T08:27:39.674320+0000","last_became_peered":"2024-03-08T08:27:39.674320+0000","last_unstale":"2024-03-08T08:27:40.665505+0000","last_undegraded":"2024-03-08T08:27:40.665505+0000","last_fullsized":"2024-03-08T08:27:40.665505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.678611+0000","last_change":"2024-03-08T08:27:39.665926+0000","last_active":"2024-03-08T08:27:39.678611+0000","last_peered":"2024-03-08T08:27:39.678611+0000","last_clean":"2024-03-08T08:27:39.678611+0000","last_became_active":"2024-03-08T08:27:39.665607+0000","last_became_peered":"2024-03-08T08:27:39.665607+0000","last_unstale":"2024-03-08T08:27:39.678611+0000","last_undegraded":"2024-03-08T08:27:39.678611+0000","last_fullsized":"2024-03-08T08:27:39.678611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.674417+0000","last_change":"2024-03-08T08:27:39.666842+0000","last_active":"2024-03-08T08:27:39.674417+0000","last_peered":"2024-03-08T08:27:39.674417+0000","last_clean":"2024-03-08T08:27:39.674417+0000","last_became_active":"2024-03-08T08:27:39.666519+0000","last_became_peered":"2024-03-08T08:27:39.666519+0000","last_unstale":"2024-03-08T08:27:39.674417+0000","last_undegraded":"2024-03-08T08:27:39.674417+0000","last_fullsized":"2024-03-08T08:27:39.674417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:22:57.654844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665072+0000","last_change":"2024-03-08T08:27:39.672195+0000","last_active":"2024-03-08T08:27:40.665072+0000","last_peered":"2024-03-08T08:27:40.665072+0000","last_clean":"2024-03-08T08:27:40.665072+0000","last_became_active":"2024-03-08T08:27:39.671927+0000","last_became_peered":"2024-03-08T08:27:39.671927+0000","last_unstale":"2024-03-08T08:27:40.665072+0000","last_undegraded":"2024-03-08T08:27:40.665072+0000","last_fullsized":"2024-03-08T08:27:40.665072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.680840+0000","last_change":"2024-03-08T08:27:39.677932+0000","last_active":"2024-03-08T08:27:39.680840+0000","last_peered":"2024-03-08T08:27:39.680840+0000","last_clean":"2024-03-08T08:27:39.680840+0000","last_became_active":"2024-03-08T08:27:39.674418+0000","last_became_peered":"2024-03-08T08:27:39.674418+0000","last_unstale":"2024-03-08T08:27:39.680840+0000","last_undegraded":"2024-03-08T08:27:39.680840+0000","last_fullsized":"2024-03-08T08:27:39.680840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683014+0000","last_change":"2024-03-08T08:27:39.680336+0000","last_active":"2024-03-08T08:27:39.683014+0000","last_peered":"2024-03-08T08:27:39.683014+0000","last_clean":"2024-03-08T08:27:39.683014+0000","last_became_active":"2024-03-08T08:27:39.680072+0000","last_became_peered":"2024-03-08T08:27:39.680072+0000","last_unstale":"2024-03-08T08:27:39.683014+0000","last_undegraded":"2024-03-08T08:27:39.683014+0000","last_fullsized":"2024-03-08T08:27:39.683014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.670156+0000","last_change":"2024-03-08T08:27:39.663330+0000","last_active":"2024-03-08T08:27:39.670156+0000","last_peered":"2024-03-08T08:27:39.670156+0000","last_clean":"2024-03-08T08:27:39.670156+0000","last_became_active":"2024-03-08T08:27:39.663063+0000","last_became_peered":"2024-03-08T08:27:39.663063+0000","last_unstale":"2024-03-08T08:27:39.670156+0000","last_undegraded":"2024-03-08T08:27:39.670156+0000","last_fullsized":"2024-03-08T08:27:39.670156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677316+0000","last_change":"2024-03-08T08:27:39.673627+0000","last_active":"2024-03-08T08:27:39.677316+0000","last_peered":"2024-03-08T08:27:39.677316+0000","last_clean":"2024-03-08T08:27:39.677316+0000","last_became_active":"2024-03-08T08:27:39.673186+0000","last_became_peered":"2024-03-08T08:27:39.673186+0000","last_unstale":"2024-03-08T08:27:39.677316+0000","last_undegraded":"2024-03-08T08:27:39.677316+0000","last_fullsized":"2024-03-08T08:27:39.677316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664828+0000","last_change":"2024-03-08T08:27:39.672747+0000","last_active":"2024-03-08T08:27:40.664828+0000","last_peered":"2024-03-08T08:27:40.664828+0000","last_clean":"2024-03-08T08:27:40.664828+0000","last_became_active":"2024-03-08T08:27:39.672478+0000","last_became_peered":"2024-03-08T08:27:39.672478+0000","last_unstale":"2024-03-08T08:27:40.664828+0000","last_undegraded":"2024-03-08T08:27:40.664828+0000","last_fullsized":"2024-03-08T08:27:40.664828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676361+0000","last_change":"2024-03-08T08:27:39.671025+0000","last_active":"2024-03-08T08:27:39.676361+0000","last_peered":"2024-03-08T08:27:39.676361+0000","last_clean":"2024-03-08T08:27:39.676361+0000","last_became_active":"2024-03-08T08:27:39.670767+0000","last_became_peered":"2024-03-08T08:27:39.670767+0000","last_unstale":"2024-03-08T08:27:39.676361+0000","last_undegraded":"2024-03-08T08:27:39.676361+0000","last_fullsized":"2024-03-08T08:27:39.676361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683030+0000","last_change":"2024-03-08T08:27:39.680602+0000","last_active":"2024-03-08T08:27:39.683030+0000","last_peered":"2024-03-08T08:27:39.683030+0000","last_clean":"2024-03-08T08:27:39.683030+0000","last_became_active":"2024-03-08T08:27:39.679691+0000","last_became_peered":"2024-03-08T08:27:39.679691+0000","last_unstale":"2024-03-08T08:27:39.683030+0000","last_undegraded":"2024-03-08T08:27:39.683030+0000","last_fullsized":"2024-03-08T08:27:39.683030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681903+0000","last_change":"2024-03-08T08:27:39.673063+0000","last_active":"2024-03-08T08:27:39.681903+0000","last_peered":"2024-03-08T08:27:39.681903+0000","last_clean":"2024-03-08T08:27:39.681903+0000","last_became_active":"2024-03-08T08:27:39.672801+0000","last_became_peered":"2024-03-08T08:27:39.672801+0000","last_unstale":"2024-03-08T08:27:39.681903+0000","last_undegraded":"2024-03-08T08:27:39.681903+0000","last_fullsized":"2024-03-08T08:27:39.681903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683008+0000","last_change":"2024-03-08T08:27:39.680324+0000","last_active":"2024-03-08T08:27:39.683008+0000","last_peered":"2024-03-08T08:27:39.683008+0000","last_clean":"2024-03-08T08:27:39.683008+0000","last_became_active":"2024-03-08T08:27:39.679988+0000","last_became_peered":"2024-03-08T08:27:39.679988+0000","last_unstale":"2024-03-08T08:27:39.683008+0000","last_undegraded":"2024-03-08T08:27:39.683008+0000","last_fullsized":"2024-03-08T08:27:39.683008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.678722+0000","last_change":"2024-03-08T08:27:39.669492+0000","last_active":"2024-03-08T08:27:39.678722+0000","last_peered":"2024-03-08T08:27:39.678722+0000","last_clean":"2024-03-08T08:27:39.678722+0000","last_became_active":"2024-03-08T08:27:39.669223+0000","last_became_peered":"2024-03-08T08:27:39.669223+0000","last_unstale":"2024-03-08T08:27:39.678722+0000","last_undegraded":"2024-03-08T08:27:39.678722+0000","last_fullsized":"2024-03-08T08:27:39.678722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665530+0000","last_change":"2024-03-08T08:27:39.675074+0000","last_active":"2024-03-08T08:27:40.665530+0000","last_peered":"2024-03-08T08:27:40.665530+0000","last_clean":"2024-03-08T08:27:40.665530+0000","last_became_active":"2024-03-08T08:27:39.674811+0000","last_became_peered":"2024-03-08T08:27:39.674811+0000","last_unstale":"2024-03-08T08:27:40.665530+0000","last_undegraded":"2024-03-08T08:27:40.665530+0000","last_fullsized":"2024-03-08T08:27:40.665530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665374+0000","last_change":"2024-03-08T08:27:39.666768+0000","last_active":"2024-03-08T08:27:40.665374+0000","last_peered":"2024-03-08T08:27:40.665374+0000","last_clean":"2024-03-08T08:27:40.665374+0000","last_became_active":"2024-03-08T08:27:39.666516+0000","last_became_peered":"2024-03-08T08:27:39.666516+0000","last_unstale":"2024-03-08T08:27:40.665374+0000","last_undegraded":"2024-03-08T08:27:40.665374+0000","last_fullsized":"2024-03-08T08:27:40.665374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681757+0000","last_change":"2024-03-08T08:27:39.672450+0000","last_active":"2024-03-08T08:27:39.681757+0000","last_peered":"2024-03-08T08:27:39.681757+0000","last_clean":"2024-03-08T08:27:39.681757+0000","last_became_active":"2024-03-08T08:27:39.671942+0000","last_became_peered":"2024-03-08T08:27:39.671942+0000","last_unstale":"2024-03-08T08:27:39.681757+0000","last_undegraded":"2024-03-08T08:27:39.681757+0000","last_fullsized":"2024-03-08T08:27:39.681757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676043+0000","last_change":"2024-03-08T08:27:39.668257+0000","last_active":"2024-03-08T08:27:39.676043+0000","last_peered":"2024-03-08T08:27:39.676043+0000","last_clean":"2024-03-08T08:27:39.676043+0000","last_became_active":"2024-03-08T08:27:39.667225+0000","last_became_peered":"2024-03-08T08:27:39.667225+0000","last_unstale":"2024-03-08T08:27:39.676043+0000","last_undegraded":"2024-03-08T08:27:39.676043+0000","last_fullsized":"2024-03-08T08:27:39.676043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:13:32.004538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683048+0000","last_change":"2024-03-08T08:27:39.680846+0000","last_active":"2024-03-08T08:27:39.683048+0000","last_peered":"2024-03-08T08:27:39.683048+0000","last_clean":"2024-03-08T08:27:39.683048+0000","last_became_active":"2024-03-08T08:27:39.680341+0000","last_became_peered":"2024-03-08T08:27:39.680341+0000","last_unstale":"2024-03-08T08:27:39.683048+0000","last_undegraded":"2024-03-08T08:27:39.683048+0000","last_fullsized":"2024-03-08T08:27:39.683048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684366+0000","last_change":"2024-03-08T08:27:39.679406+0000","last_active":"2024-03-08T08:27:39.684366+0000","last_peered":"2024-03-08T08:27:39.684366+0000","last_clean":"2024-03-08T08:27:39.684366+0000","last_became_active":"2024-03-08T08:27:39.679148+0000","last_became_peered":"2024-03-08T08:27:39.679148+0000","last_unstale":"2024-03-08T08:27:39.684366+0000","last_undegraded":"2024-03-08T08:27:39.684366+0000","last_fullsized":"2024-03-08T08:27:39.684366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664398+0000","last_change":"2024-03-08T08:27:39.670032+0000","last_active":"2024-03-08T08:27:40.664398+0000","last_peered":"2024-03-08T08:27:40.664398+0000","last_clean":"2024-03-08T08:27:40.664398+0000","last_became_active":"2024-03-08T08:27:39.669704+0000","last_became_peered":"2024-03-08T08:27:39.669704+0000","last_unstale":"2024-03-08T08:27:40.664398+0000","last_undegraded":"2024-03-08T08:27:40.664398+0000","last_fullsized":"2024-03-08T08:27:40.664398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665149+0000","last_change":"2024-03-08T08:27:39.671209+0000","last_active":"2024-03-08T08:27:40.665149+0000","last_peered":"2024-03-08T08:27:40.665149+0000","last_clean":"2024-03-08T08:27:40.665149+0000","last_became_active":"2024-03-08T08:27:39.670900+0000","last_became_peered":"2024-03-08T08:27:39.670900+0000","last_unstale":"2024-03-08T08:27:40.665149+0000","last_undegraded":"2024-03-08T08:27:40.665149+0000","last_fullsized":"2024-03-08T08:27:40.665149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681752+0000","last_change":"2024-03-08T08:27:38.661099+0000","last_active":"2024-03-08T08:27:39.681752+0000","last_peered":"2024-03-08T08:27:39.681752+0000","last_clean":"2024-03-08T08:27:39.681752+0000","last_became_active":"2024-03-08T08:27:38.660829+0000","last_became_peered":"2024-03-08T08:27:38.660829+0000","last_unstale":"2024-03-08T08:27:39.681752+0000","last_undegraded":"2024-03-08T08:27:39.681752+0000","last_fullsized":"2024-03-08T08:27:39.681752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676600+0000","last_change":"2024-03-08T08:27:39.671575+0000","last_active":"2024-03-08T08:27:39.676600+0000","last_peered":"2024-03-08T08:27:39.676600+0000","last_clean":"2024-03-08T08:27:39.676600+0000","last_became_active":"2024-03-08T08:27:39.671311+0000","last_became_peered":"2024-03-08T08:27:39.671311+0000","last_unstale":"2024-03-08T08:27:39.676600+0000","last_undegraded":"2024-03-08T08:27:39.676600+0000","last_fullsized":"2024-03-08T08:27:39.676600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664688+0000","last_change":"2024-03-08T08:27:38.651812+0000","last_active":"2024-03-08T08:27:40.664688+0000","last_peered":"2024-03-08T08:27:40.664688+0000","last_clean":"2024-03-08T08:27:40.664688+0000","last_became_active":"2024-03-08T08:27:38.651040+0000","last_became_peered":"2024-03-08T08:27:38.651040+0000","last_unstale":"2024-03-08T08:27:40.664688+0000","last_undegraded":"2024-03-08T08:27:40.664688+0000","last_fullsized":"2024-03-08T08:27:40.664688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.659208+0000","last_change":"2024-03-08T08:27:38.663033+0000","last_active":"2024-03-08T08:27:39.659208+0000","last_peered":"2024-03-08T08:27:39.659208+0000","last_clean":"2024-03-08T08:27:39.659208+0000","last_became_active":"2024-03-08T08:27:38.662753+0000","last_became_peered":"2024-03-08T08:27:38.662753+0000","last_unstale":"2024-03-08T08:27:39.659208+0000","last_undegraded":"2024-03-08T08:27:39.659208+0000","last_fullsized":"2024-03-08T08:27:39.659208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:13:30.999947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681665+0000","last_change":"2024-03-08T08:27:39.670839+0000","last_active":"2024-03-08T08:27:39.681665+0000","last_peered":"2024-03-08T08:27:39.681665+0000","last_clean":"2024-03-08T08:27:39.681665+0000","last_became_active":"2024-03-08T08:27:39.670559+0000","last_became_peered":"2024-03-08T08:27:39.670559+0000","last_unstale":"2024-03-08T08:27:39.681665+0000","last_undegraded":"2024-03-08T08:27:39.681665+0000","last_fullsized":"2024-03-08T08:27:39.681665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.669032+0000","last_change":"2024-03-08T08:27:38.656237+0000","last_active":"2024-03-08T08:27:39.669032+0000","last_peered":"2024-03-08T08:27:39.669032+0000","last_clean":"2024-03-08T08:27:39.669032+0000","last_became_active":"2024-03-08T08:27:38.655944+0000","last_became_peered":"2024-03-08T08:27:38.655944+0000","last_unstale":"2024-03-08T08:27:39.669032+0000","last_undegraded":"2024-03-08T08:27:39.669032+0000","last_fullsized":"2024-03-08T08:27:39.669032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.667881+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.666906+0000","last_became_peered":"2024-03-08T08:27:39.666906+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665029+0000","last_change":"2024-03-08T08:27:38.660261+0000","last_active":"2024-03-08T08:27:40.665029+0000","last_peered":"2024-03-08T08:27:40.665029+0000","last_clean":"2024-03-08T08:27:40.665029+0000","last_became_active":"2024-03-08T08:27:38.659979+0000","last_became_peered":"2024-03-08T08:27:38.659979+0000","last_unstale":"2024-03-08T08:27:40.665029+0000","last_undegraded":"2024-03-08T08:27:40.665029+0000","last_fullsized":"2024-03-08T08:27:40.665029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681726+0000","last_change":"2024-03-08T08:27:39.679621+0000","last_active":"2024-03-08T08:27:39.681726+0000","last_peered":"2024-03-08T08:27:39.681726+0000","last_clean":"2024-03-08T08:27:39.681726+0000","last_became_active":"2024-03-08T08:27:39.679327+0000","last_became_peered":"2024-03-08T08:27:39.679327+0000","last_unstale":"2024-03-08T08:27:39.681726+0000","last_undegraded":"2024-03-08T08:27:39.681726+0000","last_fullsized":"2024-03-08T08:27:39.681726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.669273+0000","last_change":"2024-03-08T08:27:38.662435+0000","last_active":"2024-03-08T08:27:39.669273+0000","last_peered":"2024-03-08T08:27:39.669273+0000","last_clean":"2024-03-08T08:27:39.669273+0000","last_became_active":"2024-03-08T08:27:38.662189+0000","last_became_peered":"2024-03-08T08:27:38.662189+0000","last_unstale":"2024-03-08T08:27:39.669273+0000","last_undegraded":"2024-03-08T08:27:39.669273+0000","last_fullsized":"2024-03-08T08:27:39.669273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684495+0000","last_change":"2024-03-08T08:27:39.680354+0000","last_active":"2024-03-08T08:27:39.684495+0000","last_peered":"2024-03-08T08:27:39.684495+0000","last_clean":"2024-03-08T08:27:39.684495+0000","last_became_active":"2024-03-08T08:27:39.680068+0000","last_became_peered":"2024-03-08T08:27:39.680068+0000","last_unstale":"2024-03-08T08:27:39.684495+0000","last_undegraded":"2024-03-08T08:27:39.684495+0000","last_fullsized":"2024-03-08T08:27:39.684495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665344+0000","last_change":"2024-03-08T08:27:38.662811+0000","last_active":"2024-03-08T08:27:40.665344+0000","last_peered":"2024-03-08T08:27:40.665344+0000","last_clean":"2024-03-08T08:27:40.665344+0000","last_became_active":"2024-03-08T08:27:38.662484+0000","last_became_peered":"2024-03-08T08:27:38.662484+0000","last_unstale":"2024-03-08T08:27:40.665344+0000","last_undegraded":"2024-03-08T08:27:40.665344+0000","last_fullsized":"2024-03-08T08:27:40.665344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665494+0000","last_change":"2024-03-08T08:27:39.681653+0000","last_active":"2024-03-08T08:27:40.665494+0000","last_peered":"2024-03-08T08:27:40.665494+0000","last_clean":"2024-03-08T08:27:40.665494+0000","last_became_active":"2024-03-08T08:27:39.681368+0000","last_became_peered":"2024-03-08T08:27:39.681368+0000","last_unstale":"2024-03-08T08:27:40.665494+0000","last_undegraded":"2024-03-08T08:27:40.665494+0000","last_fullsized":"2024-03-08T08:27:40.665494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664972+0000","last_change":"2024-03-08T08:27:38.655367+0000","last_active":"2024-03-08T08:27:40.664972+0000","last_peered":"2024-03-08T08:27:40.664972+0000","last_clean":"2024-03-08T08:27:40.664972+0000","last_became_active":"2024-03-08T08:27:38.655051+0000","last_became_peered":"2024-03-08T08:27:38.655051+0000","last_unstale":"2024-03-08T08:27:40.664972+0000","last_undegraded":"2024-03-08T08:27:40.664972+0000","last_fullsized":"2024-03-08T08:27:40.664972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677293+0000","last_change":"2024-03-08T08:27:39.672887+0000","last_active":"2024-03-08T08:27:39.677293+0000","last_peered":"2024-03-08T08:27:39.677293+0000","last_clean":"2024-03-08T08:27:39.677293+0000","last_became_active":"2024-03-08T08:27:39.672009+0000","last_became_peered":"2024-03-08T08:27:39.672009+0000","last_unstale":"2024-03-08T08:27:39.677293+0000","last_undegraded":"2024-03-08T08:27:39.677293+0000","last_fullsized":"2024-03-08T08:27:39.677293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683420+0000","last_change":"2024-03-08T08:27:38.650529+0000","last_active":"2024-03-08T08:27:39.683420+0000","last_peered":"2024-03-08T08:27:39.683420+0000","last_clean":"2024-03-08T08:27:39.683420+0000","last_became_active":"2024-03-08T08:27:38.650183+0000","last_became_peered":"2024-03-08T08:27:38.650183+0000","last_unstale":"2024-03-08T08:27:39.683420+0000","last_undegraded":"2024-03-08T08:27:39.683420+0000","last_fullsized":"2024-03-08T08:27:39.683420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:18:22.093931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676123+0000","last_change":"2024-03-08T08:27:39.670409+0000","last_active":"2024-03-08T08:27:39.676123+0000","last_peered":"2024-03-08T08:27:39.676123+0000","last_clean":"2024-03-08T08:27:39.676123+0000","last_became_active":"2024-03-08T08:27:39.670105+0000","last_became_peered":"2024-03-08T08:27:39.670105+0000","last_unstale":"2024-03-08T08:27:39.676123+0000","last_undegraded":"2024-03-08T08:27:39.676123+0000","last_fullsized":"2024-03-08T08:27:39.676123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.669722+0000","last_change":"2024-03-08T08:27:38.660419+0000","last_active":"2024-03-08T08:27:39.669722+0000","last_peered":"2024-03-08T08:27:39.669722+0000","last_clean":"2024-03-08T08:27:39.669722+0000","last_became_active":"2024-03-08T08:27:38.660225+0000","last_became_peered":"2024-03-08T08:27:38.660225+0000","last_unstale":"2024-03-08T08:27:39.669722+0000","last_undegraded":"2024-03-08T08:27:39.669722+0000","last_fullsized":"2024-03-08T08:27:39.669722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684470+0000","last_change":"2024-03-08T08:27:39.675352+0000","last_active":"2024-03-08T08:27:39.684470+0000","last_peered":"2024-03-08T08:27:39.684470+0000","last_clean":"2024-03-08T08:27:39.684470+0000","last_became_active":"2024-03-08T08:27:39.674881+0000","last_became_peered":"2024-03-08T08:27:39.674881+0000","last_unstale":"2024-03-08T08:27:39.684470+0000","last_undegraded":"2024-03-08T08:27:39.684470+0000","last_fullsized":"2024-03-08T08:27:39.684470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664467+0000","last_change":"2024-03-08T08:27:38.656182+0000","last_active":"2024-03-08T08:27:40.664467+0000","last_peered":"2024-03-08T08:27:40.664467+0000","last_clean":"2024-03-08T08:27:40.664467+0000","last_became_active":"2024-03-08T08:27:38.655315+0000","last_became_peered":"2024-03-08T08:27:38.655315+0000","last_unstale":"2024-03-08T08:27:40.664467+0000","last_undegraded":"2024-03-08T08:27:40.664467+0000","last_fullsized":"2024-03-08T08:27:40.664467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664532+0000","last_change":"2024-03-08T08:27:39.666898+0000","last_active":"2024-03-08T08:27:40.664532+0000","last_peered":"2024-03-08T08:27:40.664532+0000","last_clean":"2024-03-08T08:27:40.664532+0000","last_became_active":"2024-03-08T08:27:39.666563+0000","last_became_peered":"2024-03-08T08:27:39.666563+0000","last_unstale":"2024-03-08T08:27:40.664532+0000","last_undegraded":"2024-03-08T08:27:40.664532+0000","last_fullsized":"2024-03-08T08:27:40.664532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676746+0000","last_change":"2024-03-08T08:27:38.656580+0000","last_active":"2024-03-08T08:27:39.676746+0000","last_peered":"2024-03-08T08:27:39.676746+0000","last_clean":"2024-03-08T08:27:39.676746+0000","last_became_active":"2024-03-08T08:27:38.656233+0000","last_became_peered":"2024-03-08T08:27:38.656233+0000","last_unstale":"2024-03-08T08:27:39.676746+0000","last_undegraded":"2024-03-08T08:27:39.676746+0000","last_fullsized":"2024-03-08T08:27:39.676746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685643+0000","last_change":"2024-03-08T08:27:39.677499+0000","last_active":"2024-03-08T08:27:39.685643+0000","last_peered":"2024-03-08T08:27:39.685643+0000","last_clean":"2024-03-08T08:27:39.685643+0000","last_became_active":"2024-03-08T08:27:39.677213+0000","last_became_peered":"2024-03-08T08:27:39.677213+0000","last_unstale":"2024-03-08T08:27:39.685643+0000","last_undegraded":"2024-03-08T08:27:39.685643+0000","last_fullsized":"2024-03-08T08:27:39.685643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665133+0000","last_change":"2024-03-08T08:27:38.653100+0000","last_active":"2024-03-08T08:27:40.665133+0000","last_peered":"2024-03-08T08:27:40.665133+0000","last_clean":"2024-03-08T08:27:40.665133+0000","last_became_active":"2024-03-08T08:27:38.652837+0000","last_became_peered":"2024-03-08T08:27:38.652837+0000","last_unstale":"2024-03-08T08:27:40.665133+0000","last_undegraded":"2024-03-08T08:27:40.665133+0000","last_fullsized":"2024-03-08T08:27:40.665133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677348+0000","last_change":"2024-03-08T08:27:39.673698+0000","last_active":"2024-03-08T08:27:39.677348+0000","last_peered":"2024-03-08T08:27:39.677348+0000","last_clean":"2024-03-08T08:27:39.677348+0000","last_became_active":"2024-03-08T08:27:39.673424+0000","last_became_peered":"2024-03-08T08:27:39.673424+0000","last_unstale":"2024-03-08T08:27:39.677348+0000","last_undegraded":"2024-03-08T08:27:39.677348+0000","last_fullsized":"2024-03-08T08:27:39.677348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.682282+0000","last_change":"2024-03-08T08:27:38.660277+0000","last_active":"2024-03-08T08:27:39.682282+0000","last_peered":"2024-03-08T08:27:39.682282+0000","last_clean":"2024-03-08T08:27:39.682282+0000","last_became_active":"2024-03-08T08:27:38.659873+0000","last_became_peered":"2024-03-08T08:27:38.659873+0000","last_unstale":"2024-03-08T08:27:39.682282+0000","last_undegraded":"2024-03-08T08:27:39.682282+0000","last_fullsized":"2024-03-08T08:27:39.682282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677451+0000","last_change":"2024-03-08T08:27:39.674403+0000","last_active":"2024-03-08T08:27:39.677451+0000","last_peered":"2024-03-08T08:27:39.677451+0000","last_clean":"2024-03-08T08:27:39.677451+0000","last_became_active":"2024-03-08T08:27:39.674152+0000","last_became_peered":"2024-03-08T08:27:39.674152+0000","last_unstale":"2024-03-08T08:27:39.677451+0000","last_undegraded":"2024-03-08T08:27:39.677451+0000","last_fullsized":"2024-03-08T08:27:39.677451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665435+0000","last_change":"2024-03-08T08:27:38.655005+0000","last_active":"2024-03-08T08:27:40.665435+0000","last_peered":"2024-03-08T08:27:40.665435+0000","last_clean":"2024-03-08T08:27:40.665435+0000","last_became_active":"2024-03-08T08:27:38.654683+0000","last_became_peered":"2024-03-08T08:27:38.654683+0000","last_unstale":"2024-03-08T08:27:40.665435+0000","last_undegraded":"2024-03-08T08:27:40.665435+0000","last_fullsized":"2024-03-08T08:27:40.665435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681765+0000","last_change":"2024-03-08T08:27:39.673959+0000","last_active":"2024-03-08T08:27:39.681765+0000","last_peered":"2024-03-08T08:27:39.681765+0000","last_clean":"2024-03-08T08:27:39.681765+0000","last_became_active":"2024-03-08T08:27:39.673369+0000","last_became_peered":"2024-03-08T08:27:39.673369+0000","last_unstale":"2024-03-08T08:27:39.681765+0000","last_undegraded":"2024-03-08T08:27:39.681765+0000","last_fullsized":"2024-03-08T08:27:39.681765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657749+0000","last_change":"2024-03-08T08:27:38.661161+0000","last_active":"2024-03-08T08:27:39.657749+0000","last_peered":"2024-03-08T08:27:39.657749+0000","last_clean":"2024-03-08T08:27:39.657749+0000","last_became_active":"2024-03-08T08:27:38.660876+0000","last_became_peered":"2024-03-08T08:27:38.660876+0000","last_unstale":"2024-03-08T08:27:39.657749+0000","last_undegraded":"2024-03-08T08:27:39.657749+0000","last_fullsized":"2024-03-08T08:27:39.657749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685847+0000","last_change":"2024-03-08T08:27:39.682454+0000","last_active":"2024-03-08T08:27:39.685847+0000","last_peered":"2024-03-08T08:27:39.685847+0000","last_clean":"2024-03-08T08:27:39.685847+0000","last_became_active":"2024-03-08T08:27:39.682178+0000","last_became_peered":"2024-03-08T08:27:39.682178+0000","last_unstale":"2024-03-08T08:27:39.685847+0000","last_undegraded":"2024-03-08T08:27:39.685847+0000","last_fullsized":"2024-03-08T08:27:39.685847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657683+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:27:39.657683+0000","last_peered":"2024-03-08T08:27:39.657683+0000","last_clean":"2024-03-08T08:27:39.657683+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:27:39.657683+0000","last_undegraded":"2024-03-08T08:27:39.657683+0000","last_fullsized":"2024-03-08T08:27:39.657683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.668701+0000","last_change":"2024-03-08T08:27:38.659294+0000","last_active":"2024-03-08T08:27:39.668701+0000","last_peered":"2024-03-08T08:27:39.668701+0000","last_clean":"2024-03-08T08:27:39.668701+0000","last_became_active":"2024-03-08T08:27:38.659037+0000","last_became_peered":"2024-03-08T08:27:38.659037+0000","last_unstale":"2024-03-08T08:27:39.668701+0000","last_undegraded":"2024-03-08T08:27:39.668701+0000","last_fullsized":"2024-03-08T08:27:39.668701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684766+0000","last_change":"2024-03-08T08:27:39.681265+0000","last_active":"2024-03-08T08:27:39.684766+0000","last_peered":"2024-03-08T08:27:39.684766+0000","last_clean":"2024-03-08T08:27:39.684766+0000","last_became_active":"2024-03-08T08:27:39.680995+0000","last_became_peered":"2024-03-08T08:27:39.680995+0000","last_unstale":"2024-03-08T08:27:39.684766+0000","last_undegraded":"2024-03-08T08:27:39.684766+0000","last_fullsized":"2024-03-08T08:27:39.684766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657681+0000","last_change":"2024-03-08T08:27:38.663815+0000","last_active":"2024-03-08T08:27:39.657681+0000","last_peered":"2024-03-08T08:27:39.657681+0000","last_clean":"2024-03-08T08:27:39.657681+0000","last_became_active":"2024-03-08T08:27:38.663589+0000","last_became_peered":"2024-03-08T08:27:38.663589+0000","last_unstale":"2024-03-08T08:27:39.657681+0000","last_undegraded":"2024-03-08T08:27:39.657681+0000","last_fullsized":"2024-03-08T08:27:39.657681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676559+0000","last_change":"2024-03-08T08:27:39.673596+0000","last_active":"2024-03-08T08:27:39.676559+0000","last_peered":"2024-03-08T08:27:39.676559+0000","last_clean":"2024-03-08T08:27:39.676559+0000","last_became_active":"2024-03-08T08:27:39.673171+0000","last_became_peered":"2024-03-08T08:27:39.673171+0000","last_unstale":"2024-03-08T08:27:39.676559+0000","last_undegraded":"2024-03-08T08:27:39.676559+0000","last_fullsized":"2024-03-08T08:27:39.676559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657682+0000","last_change":"2024-03-08T08:27:38.658281+0000","last_active":"2024-03-08T08:27:39.657682+0000","last_peered":"2024-03-08T08:27:39.657682+0000","last_clean":"2024-03-08T08:27:39.657682+0000","last_became_active":"2024-03-08T08:27:38.657983+0000","last_became_peered":"2024-03-08T08:27:38.657983+0000","last_unstale":"2024-03-08T08:27:39.657682+0000","last_undegraded":"2024-03-08T08:27:39.657682+0000","last_fullsized":"2024-03-08T08:27:39.657682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.662315+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.661362+0000","last_became_peered":"2024-03-08T08:27:39.661362+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.669881+0000","last_change":"2024-03-08T08:27:38.659980+0000","last_active":"2024-03-08T08:27:39.669881+0000","last_peered":"2024-03-08T08:27:39.669881+0000","last_clean":"2024-03-08T08:27:39.669881+0000","last_became_active":"2024-03-08T08:27:38.659704+0000","last_became_peered":"2024-03-08T08:27:38.659704+0000","last_unstale":"2024-03-08T08:27:39.669881+0000","last_undegraded":"2024-03-08T08:27:39.669881+0000","last_fullsized":"2024-03-08T08:27:39.669881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665238+0000","last_change":"2024-03-08T08:27:39.678619+0000","last_active":"2024-03-08T08:27:40.665238+0000","last_peered":"2024-03-08T08:27:40.665238+0000","last_clean":"2024-03-08T08:27:40.665238+0000","last_became_active":"2024-03-08T08:27:39.678371+0000","last_became_peered":"2024-03-08T08:27:39.678371+0000","last_unstale":"2024-03-08T08:27:40.665238+0000","last_undegraded":"2024-03-08T08:27:40.665238+0000","last_fullsized":"2024-03-08T08:27:40.665238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665304+0000","last_change":"2024-03-08T08:27:38.654132+0000","last_active":"2024-03-08T08:27:40.665304+0000","last_peered":"2024-03-08T08:27:40.665304+0000","last_clean":"2024-03-08T08:27:40.665304+0000","last_became_active":"2024-03-08T08:27:38.653844+0000","last_became_peered":"2024-03-08T08:27:38.653844+0000","last_unstale":"2024-03-08T08:27:40.665304+0000","last_undegraded":"2024-03-08T08:27:40.665304+0000","last_fullsized":"2024-03-08T08:27:40.665304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.680984+0000","last_change":"2024-03-08T08:27:39.679299+0000","last_active":"2024-03-08T08:27:39.680984+0000","last_peered":"2024-03-08T08:27:39.680984+0000","last_clean":"2024-03-08T08:27:39.680984+0000","last_became_active":"2024-03-08T08:27:39.679023+0000","last_became_peered":"2024-03-08T08:27:39.679023+0000","last_unstale":"2024-03-08T08:27:39.680984+0000","last_undegraded":"2024-03-08T08:27:39.680984+0000","last_fullsized":"2024-03-08T08:27:39.680984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.660086+0000","last_change":"2024-03-08T08:27:38.665656+0000","last_active":"2024-03-08T08:27:39.660086+0000","last_peered":"2024-03-08T08:27:39.660086+0000","last_clean":"2024-03-08T08:27:39.660086+0000","last_became_active":"2024-03-08T08:27:38.665370+0000","last_became_peered":"2024-03-08T08:27:38.665370+0000","last_unstale":"2024-03-08T08:27:39.660086+0000","last_undegraded":"2024-03-08T08:27:39.660086+0000","last_fullsized":"2024-03-08T08:27:39.660086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665354+0000","last_change":"2024-03-08T08:27:39.664052+0000","last_active":"2024-03-08T08:27:40.665354+0000","last_peered":"2024-03-08T08:27:40.665354+0000","last_clean":"2024-03-08T08:27:40.665354+0000","last_became_active":"2024-03-08T08:27:39.663794+0000","last_became_peered":"2024-03-08T08:27:39.663794+0000","last_unstale":"2024-03-08T08:27:40.665354+0000","last_undegraded":"2024-03-08T08:27:40.665354+0000","last_fullsized":"2024-03-08T08:27:40.665354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665496+0000","last_change":"2024-03-08T08:27:38.658214+0000","last_active":"2024-03-08T08:27:40.665496+0000","last_peered":"2024-03-08T08:27:40.665496+0000","last_clean":"2024-03-08T08:27:40.665496+0000","last_became_active":"2024-03-08T08:27:38.657944+0000","last_became_peered":"2024-03-08T08:27:38.657944+0000","last_unstale":"2024-03-08T08:27:40.665496+0000","last_undegraded":"2024-03-08T08:27:40.665496+0000","last_fullsized":"2024-03-08T08:27:40.665496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681699+0000","last_change":"2024-03-08T08:27:39.671410+0000","last_active":"2024-03-08T08:27:39.681699+0000","last_peered":"2024-03-08T08:27:39.681699+0000","last_clean":"2024-03-08T08:27:39.681699+0000","last_became_active":"2024-03-08T08:27:39.671169+0000","last_became_peered":"2024-03-08T08:27:39.671169+0000","last_unstale":"2024-03-08T08:27:39.681699+0000","last_undegraded":"2024-03-08T08:27:39.681699+0000","last_fullsized":"2024-03-08T08:27:39.681699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665140+0000","last_change":"2024-03-08T08:27:38.657833+0000","last_active":"2024-03-08T08:27:40.665140+0000","last_peered":"2024-03-08T08:27:40.665140+0000","last_clean":"2024-03-08T08:27:40.665140+0000","last_became_active":"2024-03-08T08:27:38.657560+0000","last_became_peered":"2024-03-08T08:27:38.657560+0000","last_unstale":"2024-03-08T08:27:40.665140+0000","last_undegraded":"2024-03-08T08:27:40.665140+0000","last_fullsized":"2024-03-08T08:27:40.665140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.675751+0000","last_change":"2024-03-08T08:27:39.668821+0000","last_active":"2024-03-08T08:27:39.675751+0000","last_peered":"2024-03-08T08:27:39.675751+0000","last_clean":"2024-03-08T08:27:39.675751+0000","last_became_active":"2024-03-08T08:27:39.668559+0000","last_became_peered":"2024-03-08T08:27:39.668559+0000","last_unstale":"2024-03-08T08:27:39.675751+0000","last_undegraded":"2024-03-08T08:27:39.675751+0000","last_fullsized":"2024-03-08T08:27:39.675751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:26.082729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657614+0000","last_change":"2024-03-08T08:27:38.665148+0000","last_active":"2024-03-08T08:27:39.657614+0000","last_peered":"2024-03-08T08:27:39.657614+0000","last_clean":"2024-03-08T08:27:39.657614+0000","last_became_active":"2024-03-08T08:27:38.664834+0000","last_became_peered":"2024-03-08T08:27:38.664834+0000","last_unstale":"2024-03-08T08:27:39.657614+0000","last_undegraded":"2024-03-08T08:27:39.657614+0000","last_fullsized":"2024-03-08T08:27:39.657614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.667641+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.667296+0000","last_became_peered":"2024-03-08T08:27:39.667296+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665163+0000","last_change":"2024-03-08T08:27:38.660688+0000","last_active":"2024-03-08T08:27:40.665163+0000","last_peered":"2024-03-08T08:27:40.665163+0000","last_clean":"2024-03-08T08:27:40.665163+0000","last_became_active":"2024-03-08T08:27:38.660427+0000","last_became_peered":"2024-03-08T08:27:38.660427+0000","last_unstale":"2024-03-08T08:27:40.665163+0000","last_undegraded":"2024-03-08T08:27:40.665163+0000","last_fullsized":"2024-03-08T08:27:40.665163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.671217+0000","last_change":"2024-03-08T08:27:39.665202+0000","last_active":"2024-03-08T08:27:39.671217+0000","last_peered":"2024-03-08T08:27:39.671217+0000","last_clean":"2024-03-08T08:27:39.671217+0000","last_became_active":"2024-03-08T08:27:39.663567+0000","last_became_peered":"2024-03-08T08:27:39.663567+0000","last_unstale":"2024-03-08T08:27:39.671217+0000","last_undegraded":"2024-03-08T08:27:39.671217+0000","last_fullsized":"2024-03-08T08:27:39.671217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665293+0000","last_change":"2024-03-08T08:27:38.660223+0000","last_active":"2024-03-08T08:27:40.665293+0000","last_peered":"2024-03-08T08:27:40.665293+0000","last_clean":"2024-03-08T08:27:40.665293+0000","last_became_active":"2024-03-08T08:27:38.659939+0000","last_became_peered":"2024-03-08T08:27:38.659939+0000","last_unstale":"2024-03-08T08:27:40.665293+0000","last_undegraded":"2024-03-08T08:27:40.665293+0000","last_fullsized":"2024-03-08T08:27:40.665293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.678757+0000","last_change":"2024-03-08T08:27:39.670191+0000","last_active":"2024-03-08T08:27:39.678757+0000","last_peered":"2024-03-08T08:27:39.678757+0000","last_clean":"2024-03-08T08:27:39.678757+0000","last_became_active":"2024-03-08T08:27:39.669921+0000","last_became_peered":"2024-03-08T08:27:39.669921+0000","last_unstale":"2024-03-08T08:27:39.678757+0000","last_undegraded":"2024-03-08T08:27:39.678757+0000","last_fullsized":"2024-03-08T08:27:39.678757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685810+0000","last_change":"2024-03-08T08:27:38.665118+0000","last_active":"2024-03-08T08:27:39.685810+0000","last_peered":"2024-03-08T08:27:39.685810+0000","last_clean":"2024-03-08T08:27:39.685810+0000","last_became_active":"2024-03-08T08:27:38.664812+0000","last_became_peered":"2024-03-08T08:27:38.664812+0000","last_unstale":"2024-03-08T08:27:39.685810+0000","last_undegraded":"2024-03-08T08:27:39.685810+0000","last_fullsized":"2024-03-08T08:27:39.685810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664566+0000","last_change":"2024-03-08T08:27:39.672103+0000","last_active":"2024-03-08T08:27:40.664566+0000","last_peered":"2024-03-08T08:27:40.664566+0000","last_clean":"2024-03-08T08:27:40.664566+0000","last_became_active":"2024-03-08T08:27:39.671821+0000","last_became_peered":"2024-03-08T08:27:39.671821+0000","last_unstale":"2024-03-08T08:27:40.664566+0000","last_undegraded":"2024-03-08T08:27:40.664566+0000","last_fullsized":"2024-03-08T08:27:40.664566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.680899+0000","last_change":"2024-03-08T08:27:39.676821+0000","last_active":"2024-03-08T08:27:39.680899+0000","last_peered":"2024-03-08T08:27:39.680899+0000","last_clean":"2024-03-08T08:27:39.680899+0000","last_became_active":"2024-03-08T08:27:39.674571+0000","last_became_peered":"2024-03-08T08:27:39.674571+0000","last_unstale":"2024-03-08T08:27:39.680899+0000","last_undegraded":"2024-03-08T08:27:39.680899+0000","last_fullsized":"2024-03-08T08:27:39.680899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.686002+0000","last_change":"2024-03-08T08:27:38.667618+0000","last_active":"2024-03-08T08:27:39.686002+0000","last_peered":"2024-03-08T08:27:39.686002+0000","last_clean":"2024-03-08T08:27:39.686002+0000","last_became_active":"2024-03-08T08:27:38.667361+0000","last_became_peered":"2024-03-08T08:27:38.667361+0000","last_unstale":"2024-03-08T08:27:39.686002+0000","last_undegraded":"2024-03-08T08:27:39.686002+0000","last_fullsized":"2024-03-08T08:27:39.686002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.682985+0000","last_change":"2024-03-08T08:27:39.679501+0000","last_active":"2024-03-08T08:27:39.682985+0000","last_peered":"2024-03-08T08:27:39.682985+0000","last_clean":"2024-03-08T08:27:39.682985+0000","last_became_active":"2024-03-08T08:27:39.679204+0000","last_became_peered":"2024-03-08T08:27:39.679204+0000","last_unstale":"2024-03-08T08:27:39.682985+0000","last_undegraded":"2024-03-08T08:27:39.682985+0000","last_fullsized":"2024-03-08T08:27:39.682985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685796+0000","last_change":"2024-03-08T08:27:38.667616+0000","last_active":"2024-03-08T08:27:39.685796+0000","last_peered":"2024-03-08T08:27:39.685796+0000","last_clean":"2024-03-08T08:27:39.685796+0000","last_became_active":"2024-03-08T08:27:38.667335+0000","last_became_peered":"2024-03-08T08:27:38.667335+0000","last_unstale":"2024-03-08T08:27:39.685796+0000","last_undegraded":"2024-03-08T08:27:39.685796+0000","last_fullsized":"2024-03-08T08:27:39.685796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684517+0000","last_change":"2024-03-08T08:27:39.681042+0000","last_active":"2024-03-08T08:27:39.684517+0000","last_peered":"2024-03-08T08:27:39.684517+0000","last_clean":"2024-03-08T08:27:39.684517+0000","last_became_active":"2024-03-08T08:27:39.680791+0000","last_became_peered":"2024-03-08T08:27:39.680791+0000","last_unstale":"2024-03-08T08:27:39.684517+0000","last_undegraded":"2024-03-08T08:27:39.684517+0000","last_fullsized":"2024-03-08T08:27:39.684517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681792+0000","last_change":"2024-03-08T08:27:38.659145+0000","last_active":"2024-03-08T08:27:39.681792+0000","last_peered":"2024-03-08T08:27:39.681792+0000","last_clean":"2024-03-08T08:27:39.681792+0000","last_became_active":"2024-03-08T08:27:38.658848+0000","last_became_peered":"2024-03-08T08:27:38.658848+0000","last_unstale":"2024-03-08T08:27:39.681792+0000","last_undegraded":"2024-03-08T08:27:39.681792+0000","last_fullsized":"2024-03-08T08:27:39.681792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.678663+0000","last_change":"2024-03-08T08:27:39.668041+0000","last_active":"2024-03-08T08:27:39.678663+0000","last_peered":"2024-03-08T08:27:39.678663+0000","last_clean":"2024-03-08T08:27:39.678663+0000","last_became_active":"2024-03-08T08:27:39.667724+0000","last_became_peered":"2024-03-08T08:27:39.667724+0000","last_unstale":"2024-03-08T08:27:39.678663+0000","last_undegraded":"2024-03-08T08:27:39.678663+0000","last_fullsized":"2024-03-08T08:27:39.678663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.686112+0000","last_change":"2024-03-08T08:27:38.657913+0000","last_active":"2024-03-08T08:27:39.686112+0000","last_peered":"2024-03-08T08:27:39.686112+0000","last_clean":"2024-03-08T08:27:39.686112+0000","last_became_active":"2024-03-08T08:27:38.657593+0000","last_became_peered":"2024-03-08T08:27:38.657593+0000","last_unstale":"2024-03-08T08:27:39.686112+0000","last_undegraded":"2024-03-08T08:27:39.686112+0000","last_fullsized":"2024-03-08T08:27:39.686112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:27:43.297 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-08T08:27:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.431+0000 7fca43850640 1 Processor -- start 2024-03-08T08:27:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.431+0000 7fca43850640 1 -- start start 2024-03-08T08:27:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca43850640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fca3c1469e0 0x7fca3c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca43850640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0ad870 0x7fca3c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca43850640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c06dc30 con 0x7fca3c06ac80 2024-03-08T08:27:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca43850640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3c0adcd0 con 0x7fca3c1469e0 2024-03-08T08:27:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca43850640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c06deb0 con 0x7fca3c0ad870 2024-03-08T08:27:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca41ded640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fca3c06ac80 0x7fca3c0b4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57950/0 (socket says 172.21.15.105:57950) 2024-03-08T08:27:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca41ded640 1 -- 172.21.15.105:0/3247609970 learned_addr learned my addr 172.21.15.105:0/3247609970 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca40deb640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0ad870 0x7fca3c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca415ec640 1 --2- 172.21.15.105:0/3247609970 >> v2:172.21.15.144:3300/0 conn(0x7fca3c1469e0 0x7fca3c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780455802 0 0) 0x7fca34001110 con 0x7fca3c06ac80 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.432+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca28003520 con 0x7fca3c06ac80 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 103550738 0 0) 0x7fca28003520 con 0x7fca3c06ac80 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0ad870 msgr2=0x7fca3c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0ad870 0x7fca3c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 >> v2:172.21.15.144:3300/0 conn(0x7fca3c1469e0 msgr2=0x7fca3c06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 --2- 172.21.15.105:0/3247609970 >> v2:172.21.15.144:3300/0 conn(0x7fca3c1469e0 0x7fca3c06fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c06e190 con 0x7fca3c06ac80 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fca34001840 con 0x7fca3c06ac80 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 >> v1:172.21.15.105:6789/0 conn(0x7fca3c06ac80 legacy=0x7fca3c0b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca280039b0 0x7fca28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca28004800 0x7fca28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0ad870 0x7fca280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0adcd0 con 0x7fca28004800 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c06deb0 con 0x7fca280039b0 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2800b9c0 con 0x7fca3c0ad870 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca40deb640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca28004800 0x7fca28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca415ec640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca280039b0 0x7fca28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca41ded640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0ad870 0x7fca280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca40deb640 1 -- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0ad870 msgr2=0x7fca280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca40deb640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0ad870 0x7fca280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca40deb640 1 -- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca280039b0 msgr2=0x7fca28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca40deb640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca280039b0 0x7fca28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.433+0000 7fca40deb640 1 -- 172.21.15.105:0/3247609970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca28003520 con 0x7fca28004800 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.434+0000 7fca40deb640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca28004800 0x7fca28004c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009af0 tx=0x7fca28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ff3d4b3a27cd10 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.434+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c039070 con 0x7fca28004800 2024-03-08T08:27:43.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.434+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c02c6b0 con 0x7fca28004800 2024-03-08T08:27:43.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.434+0000 7fca33fff640 1 -- 172.21.15.105:0/3247609970 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c009690 con 0x7fca28004800 2024-03-08T08:27:43.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.434+0000 7fca43850640 1 -- 172.21.15.105:0/3247609970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca28004800 msgr2=0x7fca28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.434+0000 7fca43850640 1 --2- 172.21.15.105:0/3247609970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca28004800 0x7fca28004c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009af0 tx=0x7fca28005830 comp rx=0 tx=0).stop 2024-03-08T08:27:43.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.434+0000 7fca41ded640 1 -- 172.21.15.105:0/3247609970 reap_dead start 2024-03-08T08:27:43.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.435+0000 7fca43850640 1 -- 172.21.15.105:0/3247609970 shutdown_connections 2024-03-08T08:27:43.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.435+0000 7fca43850640 1 -- 172.21.15.105:0/3247609970 >> 172.21.15.105:0/3247609970 conn(0x7fca3c0b2670 msgr2=0x7fca3c0b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:43.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.435+0000 7fca43850640 1 -- 172.21.15.105:0/3247609970 shutdown_connections 2024-03-08T08:27:43.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.435+0000 7fca43850640 1 -- 172.21.15.105:0/3247609970 wait complete. 2024-03-08T08:27:43.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.436+0000 7fca43850640 1 Processor -- start 2024-03-08T08:27:43.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.436+0000 7fca43850640 1 -- start start 2024-03-08T08:27:43.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.436+0000 7fca43850640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca28004800 0x7fca3c1e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.436+0000 7fca43850640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca3c142130 0x7fca3c1e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.436+0000 7fca43850640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c1e6a30 0x7fca3c141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.436+0000 7fca43850640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c06e470 con 0x7fca3c142130 2024-03-08T08:27:43.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.436+0000 7fca43850640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b45d0 con 0x7fca28004800 2024-03-08T08:27:43.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.436+0000 7fca43850640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c146dc0 con 0x7fca3c1e6a30 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca3c142130 0x7fca3c1e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca3c142130 0x7fca3c1e5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58892/0 (socket says 172.21.15.105:58892) 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 -- 172.21.15.105:0/1811581325 learned_addr learned my addr 172.21.15.105:0/1811581325 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca41ded640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c1e6a30 0x7fca3c141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca415ec640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca28004800 0x7fca3c1e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 -- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c1e6a30 msgr2=0x7fca3c141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c1e6a30 0x7fca3c141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 -- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca28004800 msgr2=0x7fca3c1e3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca28004800 0x7fca3c1e3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 -- 172.21.15.105:0/1811581325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c1e4570 con 0x7fca3c142130 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca415ec640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca28004800 0x7fca3c1e3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca40deb640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca3c142130 0x7fca3c1e5610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fca2c037040 tx=0x7fca2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca31ffb640 1 -- 172.21.15.105:0/1811581325 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c042020 con 0x7fca3c142130 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca31ffb640 1 -- 172.21.15.105:0/1811581325 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c03a040 con 0x7fca3c142130 2024-03-08T08:27:43.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca31ffb640 1 -- 172.21.15.105:0/1811581325 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c039070 con 0x7fca3c142130 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c1e4850 con 0x7fca3c142130 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.437+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c1e4e10 con 0x7fca3c142130 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.438+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c075120 con 0x7fca3c142130 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.439+0000 7fca31ffb640 1 -- 172.21.15.105:0/1811581325 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca2c045020 con 0x7fca3c142130 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.439+0000 7fca31ffb640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fca2006b4d0 0x7fca2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.439+0000 7fca31ffb640 1 -- 172.21.15.105:0/1811581325 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca2c0ab570 con 0x7fca3c142130 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.439+0000 7fca415ec640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fca2006b4d0 0x7fca2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.440+0000 7fca415ec640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fca2006b4d0 0x7fca2006d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca38007440 tx=0x7fca3800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.443+0000 7fca31ffb640 1 -- 172.21.15.105:0/1811581325 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca2c04c070 con 0x7fca3c142130 2024-03-08T08:27:43.634 INFO:tasks.ceph.mds.d.smithi144.stderr:2024-03-08T08:27:43.633+0000 7fb72a52f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T08:27:43.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.679+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fca3c144d60 con 0x7fca3c142130 2024-03-08T08:27:43.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.679+0000 7fca31ffb640 1 -- 172.21.15.105:0/1811581325 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1011 (secure 0 0 0) 0x7fca2c07a480 con 0x7fca3c142130 2024-03-08T08:27:43.681 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:43.681 INFO:teuthology.orchestra.run.smithi105.stdout:{"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1709886446,"num_in_osds":8,"osd_in_since":1709886437,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":224149504,"bytes_avail":772869963776,"bytes_total":773094113280,"write_bytes_sec":2626,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4278}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-03-08T08:27:14.917113+0000","services":{}},"progress_events":{}} 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fca2006b4d0 msgr2=0x7fca2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fca2006b4d0 0x7fca2006d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca38007440 tx=0x7fca3800b040 comp rx=0 tx=0).stop 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca3c142130 msgr2=0x7fca3c1e5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca3c142130 0x7fca3c1e5610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fca2c037040 tx=0x7fca2c004730 comp rx=0 tx=0).stop 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 shutdown_connections 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fca2006b4d0 0x7fca2006d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c1e6a30 0x7fca3c141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca3c142130 0x7fca3c1e5610 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 --2- 172.21.15.105:0/1811581325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca28004800 0x7fca3c1e3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 >> 172.21.15.105:0/1811581325 conn(0x7fca3c0b2670 msgr2=0x7fca3c06fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 shutdown_connections 2024-03-08T08:27:43.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.681+0000 7fca43850640 1 -- 172.21.15.105:0/1811581325 wait complete. 2024-03-08T08:27:43.694 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-08T08:27:43.694 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:27:43.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.792+0000 7f678c756640 1 Processor -- start 2024-03-08T08:27:43.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.792+0000 7f678c756640 1 -- start start 2024-03-08T08:27:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.793+0000 7f678c756640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6784161850 0x7f678406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.793+0000 7f678c756640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67840d48a0 con 0x7f6784161470 2024-03-08T08:27:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.793+0000 7f678c756640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67840d4b80 con 0x7f678406bcb0 2024-03-08T08:27:43.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.793+0000 7f678c756640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678406c170 con 0x7f6784161850 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.793+0000 7f678acf3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6784161850 0x7f678406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.793+0000 7f678a4f2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6784161470 0x7f678406b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40254/0 (socket says 172.21.15.105:40254) 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.793+0000 7f678a4f2640 1 -- 172.21.15.105:0/2929906317 learned_addr learned my addr 172.21.15.105:0/2929906317 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.794+0000 7f678acf3640 1 -- 172.21.15.105:0/2929906317 >> v1:172.21.15.144:6789/0 conn(0x7f678406bcb0 legacy=0x7f67840b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.794+0000 7f678acf3640 1 -- 172.21.15.105:0/2929906317 >> v1:172.21.15.105:6789/0 conn(0x7f6784161470 legacy=0x7f678406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.794+0000 7f678acf3640 1 -- 172.21.15.105:0/2929906317 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678406c450 con 0x7f6784161850 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.794+0000 7f67894f0640 1 -- 172.21.15.105:0/2929906317 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618751801 0 0) 0x7f67840d48a0 con 0x7f6784161470 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.794+0000 7f678acf3640 1 --2- 172.21.15.105:0/2929906317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6784161850 0x7f678406c830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f677c0097a0 tx=0x7f677c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1c498919d00354 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.794+0000 7f67894f0640 1 -- 172.21.15.105:0/2929906317 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0315a0 con 0x7f6784161850 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f67894f0640 1 -- 172.21.15.105:0/2929906317 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c037070 con 0x7f6784161850 2024-03-08T08:27:43.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f67894f0640 1 -- 172.21.15.105:0/2929906317 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c031960 con 0x7f6784161850 2024-03-08T08:27:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f678c756640 1 -- 172.21.15.105:0/2929906317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6784161850 msgr2=0x7f678406c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f678c756640 1 --2- 172.21.15.105:0/2929906317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6784161850 0x7f678406c830 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f677c0097a0 tx=0x7f677c02b840 comp rx=0 tx=0).stop 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f678c756640 1 -- 172.21.15.105:0/2929906317 shutdown_connections 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f678c756640 1 --2- 172.21.15.105:0/2929906317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6784161850 0x7f678406c830 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f678c756640 1 -- 172.21.15.105:0/2929906317 >> 172.21.15.105:0/2929906317 conn(0x7f67840d3ce0 msgr2=0x7f67840d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f678c756640 1 -- 172.21.15.105:0/2929906317 shutdown_connections 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.795+0000 7f678c756640 1 -- 172.21.15.105:0/2929906317 wait complete. 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.796+0000 7f678c756640 1 Processor -- start 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.796+0000 7f678c756640 1 -- start start 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678c756640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406bcb0 0x7f67840b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678c756640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6784161470 0x7f67840b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678c756640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6784161850 0x7f67840b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678c756640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678406d4f0 con 0x7f678406bcb0 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678c756640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678406d370 con 0x7f6784161470 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678c756640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678406d670 con 0x7f6784161850 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406bcb0 0x7f67840b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406bcb0 0x7f67840b8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59196/0 (socket says 172.21.15.105:59196) 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 -- 172.21.15.105:0/2366516120 learned_addr learned my addr 172.21.15.105:0/2366516120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f6789cf1640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6784161470 0x7f67840b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678acf3640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6784161850 0x7f67840b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 -- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6784161850 msgr2=0x7f67840b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6784161850 0x7f67840b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 -- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6784161470 msgr2=0x7f67840b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6784161470 0x7f67840b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 -- 172.21.15.105:0/2366516120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840cd240 con 0x7f678406bcb0 2024-03-08T08:27:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678acf3640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6784161850 0x7f67840b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.797+0000 7f678a4f2640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406bcb0 0x7f67840b8720 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f678000c890 tx=0x7f678000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.798+0000 7f677affd640 1 -- 172.21.15.105:0/2366516120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780018070 con 0x7f678406bcb0 2024-03-08T08:27:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.798+0000 7f677affd640 1 -- 172.21.15.105:0/2366516120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678000a640 con 0x7f678406bcb0 2024-03-08T08:27:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.798+0000 7f677affd640 1 -- 172.21.15.105:0/2366516120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678000aa30 con 0x7f678406bcb0 2024-03-08T08:27:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.798+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67840b68c0 con 0x7f678406bcb0 2024-03-08T08:27:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.798+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67840b6e80 con 0x7f678406bcb0 2024-03-08T08:27:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.799+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6784075120 con 0x7f678406bcb0 2024-03-08T08:27:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.799+0000 7f677affd640 1 -- 172.21.15.105:0/2366516120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f678001b070 con 0x7f678406bcb0 2024-03-08T08:27:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.800+0000 7f677affd640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f676806b520 0x7f676806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.800+0000 7f677affd640 1 -- 172.21.15.105:0/2366516120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6780089850 con 0x7f678406bcb0 2024-03-08T08:27:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.800+0000 7f6789cf1640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f676806b520 0x7f676806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.800+0000 7f6789cf1640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f676806b520 0x7f676806d9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67840cd540 tx=0x7f6774007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:43.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.804+0000 7f677affd640 1 -- 172.21.15.105:0/2366516120 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67800581f0 con 0x7f678406bcb0 2024-03-08T08:27:43.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.972+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f678406ca20 con 0x7f676806b520 2024-03-08T08:27:43.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.978+0000 7f677affd640 1 -- 172.21.15.105:0/2366516120 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251896 (secure 0 0 0) 0x7f678406ca20 con 0x7f676806b520 2024-03-08T08:27:43.983 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:43.985 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:27:43.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f676806b520 msgr2=0x7f676806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f676806b520 0x7f676806d9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67840cd540 tx=0x7f6774007e30 comp rx=0 tx=0).stop 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406bcb0 msgr2=0x7f67840b8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406bcb0 0x7f67840b8720 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f678000c890 tx=0x7f678000cd50 comp rx=0 tx=0).stop 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 shutdown_connections 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f676806b520 0x7f676806d9e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6784161850 0x7f67840b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6784161470 0x7f67840b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 --2- 172.21.15.105:0/2366516120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f678406bcb0 0x7f67840b8720 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.981+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 >> 172.21.15.105:0/2366516120 conn(0x7f67840d3ce0 msgr2=0x7f67840d40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.982+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 shutdown_connections 2024-03-08T08:27:43.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:43.982+0000 7f678c756640 1 -- 172.21.15.105:0/2366516120 wait complete. 2024-03-08T08:27:43.994 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-08T08:27:43.096319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218896,"kb_used_data":2728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755824,"statfs":{"total":773094113280,"available":772869963776,"internally_reserved":0,"allocated":2793472,"data_stored":1702628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.458739"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.675515+0000","last_change":"2024-03-08T08:27:39.669170+0000","last_active":"2024-03-08T08:27:39.675515+0000","last_peered":"2024-03-08T08:27:39.675515+0000","last_clean":"2024-03-08T08:27:39.675515+0000","last_became_active":"2024-03-08T08:27:39.668776+0000","last_became_peered":"2024-03-08T08:27:39.668776+0000","last_unstale":"2024-03-08T08:27:39.675515+0000","last_undegraded":"2024-03-08T08:27:39.675515+0000","last_fullsized":"2024-03-08T08:27:39.675515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665542+0000","last_change":"2024-03-08T08:27:39.669937+0000","last_active":"2024-03-08T08:27:40.665542+0000","last_peered":"2024-03-08T08:27:40.665542+0000","last_clean":"2024-03-08T08:27:40.665542+0000","last_became_active":"2024-03-08T08:27:39.669643+0000","last_became_peered":"2024-03-08T08:27:39.669643+0000","last_unstale":"2024-03-08T08:27:40.665542+0000","last_undegraded":"2024-03-08T08:27:40.665542+0000","last_fullsized":"2024-03-08T08:27:40.665542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.675785+0000","last_change":"2024-03-08T08:27:39.669698+0000","last_active":"2024-03-08T08:27:39.675785+0000","last_peered":"2024-03-08T08:27:39.675785+0000","last_clean":"2024-03-08T08:27:39.675785+0000","last_became_active":"2024-03-08T08:27:39.669412+0000","last_became_peered":"2024-03-08T08:27:39.669412+0000","last_unstale":"2024-03-08T08:27:39.675785+0000","last_undegraded":"2024-03-08T08:27:39.675785+0000","last_fullsized":"2024-03-08T08:27:39.675785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:16.844234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681784+0000","last_change":"2024-03-08T08:27:39.674216+0000","last_active":"2024-03-08T08:27:39.681784+0000","last_peered":"2024-03-08T08:27:39.681784+0000","last_clean":"2024-03-08T08:27:39.681784+0000","last_became_active":"2024-03-08T08:27:39.673664+0000","last_became_peered":"2024-03-08T08:27:39.673664+0000","last_unstale":"2024-03-08T08:27:39.681784+0000","last_undegraded":"2024-03-08T08:27:39.681784+0000","last_fullsized":"2024-03-08T08:27:39.681784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.675489+0000","last_change":"2024-03-08T08:27:39.671956+0000","last_active":"2024-03-08T08:27:39.675489+0000","last_peered":"2024-03-08T08:27:39.675489+0000","last_clean":"2024-03-08T08:27:39.675489+0000","last_became_active":"2024-03-08T08:27:39.671671+0000","last_became_peered":"2024-03-08T08:27:39.671671+0000","last_unstale":"2024-03-08T08:27:39.675489+0000","last_undegraded":"2024-03-08T08:27:39.675489+0000","last_fullsized":"2024-03-08T08:27:39.675489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684558+0000","last_change":"2024-03-08T08:27:39.681569+0000","last_active":"2024-03-08T08:27:39.684558+0000","last_peered":"2024-03-08T08:27:39.684558+0000","last_clean":"2024-03-08T08:27:39.684558+0000","last_became_active":"2024-03-08T08:27:39.681306+0000","last_became_peered":"2024-03-08T08:27:39.681306+0000","last_unstale":"2024-03-08T08:27:39.684558+0000","last_undegraded":"2024-03-08T08:27:39.684558+0000","last_fullsized":"2024-03-08T08:27:39.684558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677011+0000","last_change":"2024-03-08T08:27:39.669471+0000","last_active":"2024-03-08T08:27:39.677011+0000","last_peered":"2024-03-08T08:27:39.677011+0000","last_clean":"2024-03-08T08:27:39.677011+0000","last_became_active":"2024-03-08T08:27:39.669144+0000","last_became_peered":"2024-03-08T08:27:39.669144+0000","last_unstale":"2024-03-08T08:27:39.677011+0000","last_undegraded":"2024-03-08T08:27:39.677011+0000","last_fullsized":"2024-03-08T08:27:39.677011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684664+0000","last_change":"2024-03-08T08:27:39.680587+0000","last_active":"2024-03-08T08:27:39.684664+0000","last_peered":"2024-03-08T08:27:39.684664+0000","last_clean":"2024-03-08T08:27:39.684664+0000","last_became_active":"2024-03-08T08:27:39.680354+0000","last_became_peered":"2024-03-08T08:27:39.680354+0000","last_unstale":"2024-03-08T08:27:39.684664+0000","last_undegraded":"2024-03-08T08:27:39.684664+0000","last_fullsized":"2024-03-08T08:27:39.684664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685850+0000","last_change":"2024-03-08T08:27:39.682744+0000","last_active":"2024-03-08T08:27:39.685850+0000","last_peered":"2024-03-08T08:27:39.685850+0000","last_clean":"2024-03-08T08:27:39.685850+0000","last_became_active":"2024-03-08T08:27:39.682486+0000","last_became_peered":"2024-03-08T08:27:39.682486+0000","last_unstale":"2024-03-08T08:27:39.685850+0000","last_undegraded":"2024-03-08T08:27:39.685850+0000","last_fullsized":"2024-03-08T08:27:39.685850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665493+0000","last_change":"2024-03-08T08:27:39.681720+0000","last_active":"2024-03-08T08:27:40.665493+0000","last_peered":"2024-03-08T08:27:40.665493+0000","last_clean":"2024-03-08T08:27:40.665493+0000","last_became_active":"2024-03-08T08:27:39.681473+0000","last_became_peered":"2024-03-08T08:27:39.681473+0000","last_unstale":"2024-03-08T08:27:40.665493+0000","last_undegraded":"2024-03-08T08:27:40.665493+0000","last_fullsized":"2024-03-08T08:27:40.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.681092+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.680824+0000","last_became_peered":"2024-03-08T08:27:39.680824+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665505+0000","last_change":"2024-03-08T08:27:39.674803+0000","last_active":"2024-03-08T08:27:40.665505+0000","last_peered":"2024-03-08T08:27:40.665505+0000","last_clean":"2024-03-08T08:27:40.665505+0000","last_became_active":"2024-03-08T08:27:39.674320+0000","last_became_peered":"2024-03-08T08:27:39.674320+0000","last_unstale":"2024-03-08T08:27:40.665505+0000","last_undegraded":"2024-03-08T08:27:40.665505+0000","last_fullsized":"2024-03-08T08:27:40.665505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.678611+0000","last_change":"2024-03-08T08:27:39.665926+0000","last_active":"2024-03-08T08:27:39.678611+0000","last_peered":"2024-03-08T08:27:39.678611+0000","last_clean":"2024-03-08T08:27:39.678611+0000","last_became_active":"2024-03-08T08:27:39.665607+0000","last_became_peered":"2024-03-08T08:27:39.665607+0000","last_unstale":"2024-03-08T08:27:39.678611+0000","last_undegraded":"2024-03-08T08:27:39.678611+0000","last_fullsized":"2024-03-08T08:27:39.678611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.674417+0000","last_change":"2024-03-08T08:27:39.666842+0000","last_active":"2024-03-08T08:27:39.674417+0000","last_peered":"2024-03-08T08:27:39.674417+0000","last_clean":"2024-03-08T08:27:39.674417+0000","last_became_active":"2024-03-08T08:27:39.666519+0000","last_became_peered":"2024-03-08T08:27:39.666519+0000","last_unstale":"2024-03-08T08:27:39.674417+0000","last_undegraded":"2024-03-08T08:27:39.674417+0000","last_fullsized":"2024-03-08T08:27:39.674417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:22:57.654844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665072+0000","last_change":"2024-03-08T08:27:39.672195+0000","last_active":"2024-03-08T08:27:40.665072+0000","last_peered":"2024-03-08T08:27:40.665072+0000","last_clean":"2024-03-08T08:27:40.665072+0000","last_became_active":"2024-03-08T08:27:39.671927+0000","last_became_peered":"2024-03-08T08:27:39.671927+0000","last_unstale":"2024-03-08T08:27:40.665072+0000","last_undegraded":"2024-03-08T08:27:40.665072+0000","last_fullsized":"2024-03-08T08:27:40.665072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.680840+0000","last_change":"2024-03-08T08:27:39.677932+0000","last_active":"2024-03-08T08:27:39.680840+0000","last_peered":"2024-03-08T08:27:39.680840+0000","last_clean":"2024-03-08T08:27:39.680840+0000","last_became_active":"2024-03-08T08:27:39.674418+0000","last_became_peered":"2024-03-08T08:27:39.674418+0000","last_unstale":"2024-03-08T08:27:39.680840+0000","last_undegraded":"2024-03-08T08:27:39.680840+0000","last_fullsized":"2024-03-08T08:27:39.680840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683014+0000","last_change":"2024-03-08T08:27:39.680336+0000","last_active":"2024-03-08T08:27:39.683014+0000","last_peered":"2024-03-08T08:27:39.683014+0000","last_clean":"2024-03-08T08:27:39.683014+0000","last_became_active":"2024-03-08T08:27:39.680072+0000","last_became_peered":"2024-03-08T08:27:39.680072+0000","last_unstale":"2024-03-08T08:27:39.683014+0000","last_undegraded":"2024-03-08T08:27:39.683014+0000","last_fullsized":"2024-03-08T08:27:39.683014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.670156+0000","last_change":"2024-03-08T08:27:39.663330+0000","last_active":"2024-03-08T08:27:39.670156+0000","last_peered":"2024-03-08T08:27:39.670156+0000","last_clean":"2024-03-08T08:27:39.670156+0000","last_became_active":"2024-03-08T08:27:39.663063+0000","last_became_peered":"2024-03-08T08:27:39.663063+0000","last_unstale":"2024-03-08T08:27:39.670156+0000","last_undegraded":"2024-03-08T08:27:39.670156+0000","last_fullsized":"2024-03-08T08:27:39.670156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677316+0000","last_change":"2024-03-08T08:27:39.673627+0000","last_active":"2024-03-08T08:27:39.677316+0000","last_peered":"2024-03-08T08:27:39.677316+0000","last_clean":"2024-03-08T08:27:39.677316+0000","last_became_active":"2024-03-08T08:27:39.673186+0000","last_became_peered":"2024-03-08T08:27:39.673186+0000","last_unstale":"2024-03-08T08:27:39.677316+0000","last_undegraded":"2024-03-08T08:27:39.677316+0000","last_fullsized":"2024-03-08T08:27:39.677316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664828+0000","last_change":"2024-03-08T08:27:39.672747+0000","last_active":"2024-03-08T08:27:40.664828+0000","last_peered":"2024-03-08T08:27:40.664828+0000","last_clean":"2024-03-08T08:27:40.664828+0000","last_became_active":"2024-03-08T08:27:39.672478+0000","last_became_peered":"2024-03-08T08:27:39.672478+0000","last_unstale":"2024-03-08T08:27:40.664828+0000","last_undegraded":"2024-03-08T08:27:40.664828+0000","last_fullsized":"2024-03-08T08:27:40.664828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676361+0000","last_change":"2024-03-08T08:27:39.671025+0000","last_active":"2024-03-08T08:27:39.676361+0000","last_peered":"2024-03-08T08:27:39.676361+0000","last_clean":"2024-03-08T08:27:39.676361+0000","last_became_active":"2024-03-08T08:27:39.670767+0000","last_became_peered":"2024-03-08T08:27:39.670767+0000","last_unstale":"2024-03-08T08:27:39.676361+0000","last_undegraded":"2024-03-08T08:27:39.676361+0000","last_fullsized":"2024-03-08T08:27:39.676361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683030+0000","last_change":"2024-03-08T08:27:39.680602+0000","last_active":"2024-03-08T08:27:39.683030+0000","last_peered":"2024-03-08T08:27:39.683030+0000","last_clean":"2024-03-08T08:27:39.683030+0000","last_became_active":"2024-03-08T08:27:39.679691+0000","last_became_peered":"2024-03-08T08:27:39.679691+0000","last_unstale":"2024-03-08T08:27:39.683030+0000","last_undegraded":"2024-03-08T08:27:39.683030+0000","last_fullsized":"2024-03-08T08:27:39.683030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681903+0000","last_change":"2024-03-08T08:27:39.673063+0000","last_active":"2024-03-08T08:27:39.681903+0000","last_peered":"2024-03-08T08:27:39.681903+0000","last_clean":"2024-03-08T08:27:39.681903+0000","last_became_active":"2024-03-08T08:27:39.672801+0000","last_became_peered":"2024-03-08T08:27:39.672801+0000","last_unstale":"2024-03-08T08:27:39.681903+0000","last_undegraded":"2024-03-08T08:27:39.681903+0000","last_fullsized":"2024-03-08T08:27:39.681903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683008+0000","last_change":"2024-03-08T08:27:39.680324+0000","last_active":"2024-03-08T08:27:39.683008+0000","last_peered":"2024-03-08T08:27:39.683008+0000","last_clean":"2024-03-08T08:27:39.683008+0000","last_became_active":"2024-03-08T08:27:39.679988+0000","last_became_peered":"2024-03-08T08:27:39.679988+0000","last_unstale":"2024-03-08T08:27:39.683008+0000","last_undegraded":"2024-03-08T08:27:39.683008+0000","last_fullsized":"2024-03-08T08:27:39.683008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.678722+0000","last_change":"2024-03-08T08:27:39.669492+0000","last_active":"2024-03-08T08:27:39.678722+0000","last_peered":"2024-03-08T08:27:39.678722+0000","last_clean":"2024-03-08T08:27:39.678722+0000","last_became_active":"2024-03-08T08:27:39.669223+0000","last_became_peered":"2024-03-08T08:27:39.669223+0000","last_unstale":"2024-03-08T08:27:39.678722+0000","last_undegraded":"2024-03-08T08:27:39.678722+0000","last_fullsized":"2024-03-08T08:27:39.678722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665530+0000","last_change":"2024-03-08T08:27:39.675074+0000","last_active":"2024-03-08T08:27:40.665530+0000","last_peered":"2024-03-08T08:27:40.665530+0000","last_clean":"2024-03-08T08:27:40.665530+0000","last_became_active":"2024-03-08T08:27:39.674811+0000","last_became_peered":"2024-03-08T08:27:39.674811+0000","last_unstale":"2024-03-08T08:27:40.665530+0000","last_undegraded":"2024-03-08T08:27:40.665530+0000","last_fullsized":"2024-03-08T08:27:40.665530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665374+0000","last_change":"2024-03-08T08:27:39.666768+0000","last_active":"2024-03-08T08:27:40.665374+0000","last_peered":"2024-03-08T08:27:40.665374+0000","last_clean":"2024-03-08T08:27:40.665374+0000","last_became_active":"2024-03-08T08:27:39.666516+0000","last_became_peered":"2024-03-08T08:27:39.666516+0000","last_unstale":"2024-03-08T08:27:40.665374+0000","last_undegraded":"2024-03-08T08:27:40.665374+0000","last_fullsized":"2024-03-08T08:27:40.665374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681757+0000","last_change":"2024-03-08T08:27:39.672450+0000","last_active":"2024-03-08T08:27:39.681757+0000","last_peered":"2024-03-08T08:27:39.681757+0000","last_clean":"2024-03-08T08:27:39.681757+0000","last_became_active":"2024-03-08T08:27:39.671942+0000","last_became_peered":"2024-03-08T08:27:39.671942+0000","last_unstale":"2024-03-08T08:27:39.681757+0000","last_undegraded":"2024-03-08T08:27:39.681757+0000","last_fullsized":"2024-03-08T08:27:39.681757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676043+0000","last_change":"2024-03-08T08:27:39.668257+0000","last_active":"2024-03-08T08:27:39.676043+0000","last_peered":"2024-03-08T08:27:39.676043+0000","last_clean":"2024-03-08T08:27:39.676043+0000","last_became_active":"2024-03-08T08:27:39.667225+0000","last_became_peered":"2024-03-08T08:27:39.667225+0000","last_unstale":"2024-03-08T08:27:39.676043+0000","last_undegraded":"2024-03-08T08:27:39.676043+0000","last_fullsized":"2024-03-08T08:27:39.676043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:13:32.004538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683048+0000","last_change":"2024-03-08T08:27:39.680846+0000","last_active":"2024-03-08T08:27:39.683048+0000","last_peered":"2024-03-08T08:27:39.683048+0000","last_clean":"2024-03-08T08:27:39.683048+0000","last_became_active":"2024-03-08T08:27:39.680341+0000","last_became_peered":"2024-03-08T08:27:39.680341+0000","last_unstale":"2024-03-08T08:27:39.683048+0000","last_undegraded":"2024-03-08T08:27:39.683048+0000","last_fullsized":"2024-03-08T08:27:39.683048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684366+0000","last_change":"2024-03-08T08:27:39.679406+0000","last_active":"2024-03-08T08:27:39.684366+0000","last_peered":"2024-03-08T08:27:39.684366+0000","last_clean":"2024-03-08T08:27:39.684366+0000","last_became_active":"2024-03-08T08:27:39.679148+0000","last_became_peered":"2024-03-08T08:27:39.679148+0000","last_unstale":"2024-03-08T08:27:39.684366+0000","last_undegraded":"2024-03-08T08:27:39.684366+0000","last_fullsized":"2024-03-08T08:27:39.684366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664398+0000","last_change":"2024-03-08T08:27:39.670032+0000","last_active":"2024-03-08T08:27:40.664398+0000","last_peered":"2024-03-08T08:27:40.664398+0000","last_clean":"2024-03-08T08:27:40.664398+0000","last_became_active":"2024-03-08T08:27:39.669704+0000","last_became_peered":"2024-03-08T08:27:39.669704+0000","last_unstale":"2024-03-08T08:27:40.664398+0000","last_undegraded":"2024-03-08T08:27:40.664398+0000","last_fullsized":"2024-03-08T08:27:40.664398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665149+0000","last_change":"2024-03-08T08:27:39.671209+0000","last_active":"2024-03-08T08:27:40.665149+0000","last_peered":"2024-03-08T08:27:40.665149+0000","last_clean":"2024-03-08T08:27:40.665149+0000","last_became_active":"2024-03-08T08:27:39.670900+0000","last_became_peered":"2024-03-08T08:27:39.670900+0000","last_unstale":"2024-03-08T08:27:40.665149+0000","last_undegraded":"2024-03-08T08:27:40.665149+0000","last_fullsized":"2024-03-08T08:27:40.665149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681752+0000","last_change":"2024-03-08T08:27:38.661099+0000","last_active":"2024-03-08T08:27:39.681752+0000","last_peered":"2024-03-08T08:27:39.681752+0000","last_clean":"2024-03-08T08:27:39.681752+0000","last_became_active":"2024-03-08T08:27:38.660829+0000","last_became_peered":"2024-03-08T08:27:38.660829+0000","last_unstale":"2024-03-08T08:27:39.681752+0000","last_undegraded":"2024-03-08T08:27:39.681752+0000","last_fullsized":"2024-03-08T08:27:39.681752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676600+0000","last_change":"2024-03-08T08:27:39.671575+0000","last_active":"2024-03-08T08:27:39.676600+0000","last_peered":"2024-03-08T08:27:39.676600+0000","last_clean":"2024-03-08T08:27:39.676600+0000","last_became_active":"2024-03-08T08:27:39.671311+0000","last_became_peered":"2024-03-08T08:27:39.671311+0000","last_unstale":"2024-03-08T08:27:39.676600+0000","last_undegraded":"2024-03-08T08:27:39.676600+0000","last_fullsized":"2024-03-08T08:27:39.676600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664688+0000","last_change":"2024-03-08T08:27:38.651812+0000","last_active":"2024-03-08T08:27:40.664688+0000","last_peered":"2024-03-08T08:27:40.664688+0000","last_clean":"2024-03-08T08:27:40.664688+0000","last_became_active":"2024-03-08T08:27:38.651040+0000","last_became_peered":"2024-03-08T08:27:38.651040+0000","last_unstale":"2024-03-08T08:27:40.664688+0000","last_undegraded":"2024-03-08T08:27:40.664688+0000","last_fullsized":"2024-03-08T08:27:40.664688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.659208+0000","last_change":"2024-03-08T08:27:38.663033+0000","last_active":"2024-03-08T08:27:39.659208+0000","last_peered":"2024-03-08T08:27:39.659208+0000","last_clean":"2024-03-08T08:27:39.659208+0000","last_became_active":"2024-03-08T08:27:38.662753+0000","last_became_peered":"2024-03-08T08:27:38.662753+0000","last_unstale":"2024-03-08T08:27:39.659208+0000","last_undegraded":"2024-03-08T08:27:39.659208+0000","last_fullsized":"2024-03-08T08:27:39.659208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:13:30.999947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681665+0000","last_change":"2024-03-08T08:27:39.670839+0000","last_active":"2024-03-08T08:27:39.681665+0000","last_peered":"2024-03-08T08:27:39.681665+0000","last_clean":"2024-03-08T08:27:39.681665+0000","last_became_active":"2024-03-08T08:27:39.670559+0000","last_became_peered":"2024-03-08T08:27:39.670559+0000","last_unstale":"2024-03-08T08:27:39.681665+0000","last_undegraded":"2024-03-08T08:27:39.681665+0000","last_fullsized":"2024-03-08T08:27:39.681665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.669032+0000","last_change":"2024-03-08T08:27:38.656237+0000","last_active":"2024-03-08T08:27:39.669032+0000","last_peered":"2024-03-08T08:27:39.669032+0000","last_clean":"2024-03-08T08:27:39.669032+0000","last_became_active":"2024-03-08T08:27:38.655944+0000","last_became_peered":"2024-03-08T08:27:38.655944+0000","last_unstale":"2024-03-08T08:27:39.669032+0000","last_undegraded":"2024-03-08T08:27:39.669032+0000","last_fullsized":"2024-03-08T08:27:39.669032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.667881+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.666906+0000","last_became_peered":"2024-03-08T08:27:39.666906+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665029+0000","last_change":"2024-03-08T08:27:38.660261+0000","last_active":"2024-03-08T08:27:40.665029+0000","last_peered":"2024-03-08T08:27:40.665029+0000","last_clean":"2024-03-08T08:27:40.665029+0000","last_became_active":"2024-03-08T08:27:38.659979+0000","last_became_peered":"2024-03-08T08:27:38.659979+0000","last_unstale":"2024-03-08T08:27:40.665029+0000","last_undegraded":"2024-03-08T08:27:40.665029+0000","last_fullsized":"2024-03-08T08:27:40.665029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681726+0000","last_change":"2024-03-08T08:27:39.679621+0000","last_active":"2024-03-08T08:27:39.681726+0000","last_peered":"2024-03-08T08:27:39.681726+0000","last_clean":"2024-03-08T08:27:39.681726+0000","last_became_active":"2024-03-08T08:27:39.679327+0000","last_became_peered":"2024-03-08T08:27:39.679327+0000","last_unstale":"2024-03-08T08:27:39.681726+0000","last_undegraded":"2024-03-08T08:27:39.681726+0000","last_fullsized":"2024-03-08T08:27:39.681726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.669273+0000","last_change":"2024-03-08T08:27:38.662435+0000","last_active":"2024-03-08T08:27:39.669273+0000","last_peered":"2024-03-08T08:27:39.669273+0000","last_clean":"2024-03-08T08:27:39.669273+0000","last_became_active":"2024-03-08T08:27:38.662189+0000","last_became_peered":"2024-03-08T08:27:38.662189+0000","last_unstale":"2024-03-08T08:27:39.669273+0000","last_undegraded":"2024-03-08T08:27:39.669273+0000","last_fullsized":"2024-03-08T08:27:39.669273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684495+0000","last_change":"2024-03-08T08:27:39.680354+0000","last_active":"2024-03-08T08:27:39.684495+0000","last_peered":"2024-03-08T08:27:39.684495+0000","last_clean":"2024-03-08T08:27:39.684495+0000","last_became_active":"2024-03-08T08:27:39.680068+0000","last_became_peered":"2024-03-08T08:27:39.680068+0000","last_unstale":"2024-03-08T08:27:39.684495+0000","last_undegraded":"2024-03-08T08:27:39.684495+0000","last_fullsized":"2024-03-08T08:27:39.684495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665344+0000","last_change":"2024-03-08T08:27:38.662811+0000","last_active":"2024-03-08T08:27:40.665344+0000","last_peered":"2024-03-08T08:27:40.665344+0000","last_clean":"2024-03-08T08:27:40.665344+0000","last_became_active":"2024-03-08T08:27:38.662484+0000","last_became_peered":"2024-03-08T08:27:38.662484+0000","last_unstale":"2024-03-08T08:27:40.665344+0000","last_undegraded":"2024-03-08T08:27:40.665344+0000","last_fullsized":"2024-03-08T08:27:40.665344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665494+0000","last_change":"2024-03-08T08:27:39.681653+0000","last_active":"2024-03-08T08:27:40.665494+0000","last_peered":"2024-03-08T08:27:40.665494+0000","last_clean":"2024-03-08T08:27:40.665494+0000","last_became_active":"2024-03-08T08:27:39.681368+0000","last_became_peered":"2024-03-08T08:27:39.681368+0000","last_unstale":"2024-03-08T08:27:40.665494+0000","last_undegraded":"2024-03-08T08:27:40.665494+0000","last_fullsized":"2024-03-08T08:27:40.665494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664972+0000","last_change":"2024-03-08T08:27:38.655367+0000","last_active":"2024-03-08T08:27:40.664972+0000","last_peered":"2024-03-08T08:27:40.664972+0000","last_clean":"2024-03-08T08:27:40.664972+0000","last_became_active":"2024-03-08T08:27:38.655051+0000","last_became_peered":"2024-03-08T08:27:38.655051+0000","last_unstale":"2024-03-08T08:27:40.664972+0000","last_undegraded":"2024-03-08T08:27:40.664972+0000","last_fullsized":"2024-03-08T08:27:40.664972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677293+0000","last_change":"2024-03-08T08:27:39.672887+0000","last_active":"2024-03-08T08:27:39.677293+0000","last_peered":"2024-03-08T08:27:39.677293+0000","last_clean":"2024-03-08T08:27:39.677293+0000","last_became_active":"2024-03-08T08:27:39.672009+0000","last_became_peered":"2024-03-08T08:27:39.672009+0000","last_unstale":"2024-03-08T08:27:39.677293+0000","last_undegraded":"2024-03-08T08:27:39.677293+0000","last_fullsized":"2024-03-08T08:27:39.677293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.683420+0000","last_change":"2024-03-08T08:27:38.650529+0000","last_active":"2024-03-08T08:27:39.683420+0000","last_peered":"2024-03-08T08:27:39.683420+0000","last_clean":"2024-03-08T08:27:39.683420+0000","last_became_active":"2024-03-08T08:27:38.650183+0000","last_became_peered":"2024-03-08T08:27:38.650183+0000","last_unstale":"2024-03-08T08:27:39.683420+0000","last_undegraded":"2024-03-08T08:27:39.683420+0000","last_fullsized":"2024-03-08T08:27:39.683420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:18:22.093931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676123+0000","last_change":"2024-03-08T08:27:39.670409+0000","last_active":"2024-03-08T08:27:39.676123+0000","last_peered":"2024-03-08T08:27:39.676123+0000","last_clean":"2024-03-08T08:27:39.676123+0000","last_became_active":"2024-03-08T08:27:39.670105+0000","last_became_peered":"2024-03-08T08:27:39.670105+0000","last_unstale":"2024-03-08T08:27:39.676123+0000","last_undegraded":"2024-03-08T08:27:39.676123+0000","last_fullsized":"2024-03-08T08:27:39.676123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.669722+0000","last_change":"2024-03-08T08:27:38.660419+0000","last_active":"2024-03-08T08:27:39.669722+0000","last_peered":"2024-03-08T08:27:39.669722+0000","last_clean":"2024-03-08T08:27:39.669722+0000","last_became_active":"2024-03-08T08:27:38.660225+0000","last_became_peered":"2024-03-08T08:27:38.660225+0000","last_unstale":"2024-03-08T08:27:39.669722+0000","last_undegraded":"2024-03-08T08:27:39.669722+0000","last_fullsized":"2024-03-08T08:27:39.669722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684470+0000","last_change":"2024-03-08T08:27:39.675352+0000","last_active":"2024-03-08T08:27:39.684470+0000","last_peered":"2024-03-08T08:27:39.684470+0000","last_clean":"2024-03-08T08:27:39.684470+0000","last_became_active":"2024-03-08T08:27:39.674881+0000","last_became_peered":"2024-03-08T08:27:39.674881+0000","last_unstale":"2024-03-08T08:27:39.684470+0000","last_undegraded":"2024-03-08T08:27:39.684470+0000","last_fullsized":"2024-03-08T08:27:39.684470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664467+0000","last_change":"2024-03-08T08:27:38.656182+0000","last_active":"2024-03-08T08:27:40.664467+0000","last_peered":"2024-03-08T08:27:40.664467+0000","last_clean":"2024-03-08T08:27:40.664467+0000","last_became_active":"2024-03-08T08:27:38.655315+0000","last_became_peered":"2024-03-08T08:27:38.655315+0000","last_unstale":"2024-03-08T08:27:40.664467+0000","last_undegraded":"2024-03-08T08:27:40.664467+0000","last_fullsized":"2024-03-08T08:27:40.664467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664532+0000","last_change":"2024-03-08T08:27:39.666898+0000","last_active":"2024-03-08T08:27:40.664532+0000","last_peered":"2024-03-08T08:27:40.664532+0000","last_clean":"2024-03-08T08:27:40.664532+0000","last_became_active":"2024-03-08T08:27:39.666563+0000","last_became_peered":"2024-03-08T08:27:39.666563+0000","last_unstale":"2024-03-08T08:27:40.664532+0000","last_undegraded":"2024-03-08T08:27:40.664532+0000","last_fullsized":"2024-03-08T08:27:40.664532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676746+0000","last_change":"2024-03-08T08:27:38.656580+0000","last_active":"2024-03-08T08:27:39.676746+0000","last_peered":"2024-03-08T08:27:39.676746+0000","last_clean":"2024-03-08T08:27:39.676746+0000","last_became_active":"2024-03-08T08:27:38.656233+0000","last_became_peered":"2024-03-08T08:27:38.656233+0000","last_unstale":"2024-03-08T08:27:39.676746+0000","last_undegraded":"2024-03-08T08:27:39.676746+0000","last_fullsized":"2024-03-08T08:27:39.676746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685643+0000","last_change":"2024-03-08T08:27:39.677499+0000","last_active":"2024-03-08T08:27:39.685643+0000","last_peered":"2024-03-08T08:27:39.685643+0000","last_clean":"2024-03-08T08:27:39.685643+0000","last_became_active":"2024-03-08T08:27:39.677213+0000","last_became_peered":"2024-03-08T08:27:39.677213+0000","last_unstale":"2024-03-08T08:27:39.685643+0000","last_undegraded":"2024-03-08T08:27:39.685643+0000","last_fullsized":"2024-03-08T08:27:39.685643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665133+0000","last_change":"2024-03-08T08:27:38.653100+0000","last_active":"2024-03-08T08:27:40.665133+0000","last_peered":"2024-03-08T08:27:40.665133+0000","last_clean":"2024-03-08T08:27:40.665133+0000","last_became_active":"2024-03-08T08:27:38.652837+0000","last_became_peered":"2024-03-08T08:27:38.652837+0000","last_unstale":"2024-03-08T08:27:40.665133+0000","last_undegraded":"2024-03-08T08:27:40.665133+0000","last_fullsized":"2024-03-08T08:27:40.665133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677348+0000","last_change":"2024-03-08T08:27:39.673698+0000","last_active":"2024-03-08T08:27:39.677348+0000","last_peered":"2024-03-08T08:27:39.677348+0000","last_clean":"2024-03-08T08:27:39.677348+0000","last_became_active":"2024-03-08T08:27:39.673424+0000","last_became_peered":"2024-03-08T08:27:39.673424+0000","last_unstale":"2024-03-08T08:27:39.677348+0000","last_undegraded":"2024-03-08T08:27:39.677348+0000","last_fullsized":"2024-03-08T08:27:39.677348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.682282+0000","last_change":"2024-03-08T08:27:38.660277+0000","last_active":"2024-03-08T08:27:39.682282+0000","last_peered":"2024-03-08T08:27:39.682282+0000","last_clean":"2024-03-08T08:27:39.682282+0000","last_became_active":"2024-03-08T08:27:38.659873+0000","last_became_peered":"2024-03-08T08:27:38.659873+0000","last_unstale":"2024-03-08T08:27:39.682282+0000","last_undegraded":"2024-03-08T08:27:39.682282+0000","last_fullsized":"2024-03-08T08:27:39.682282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.677451+0000","last_change":"2024-03-08T08:27:39.674403+0000","last_active":"2024-03-08T08:27:39.677451+0000","last_peered":"2024-03-08T08:27:39.677451+0000","last_clean":"2024-03-08T08:27:39.677451+0000","last_became_active":"2024-03-08T08:27:39.674152+0000","last_became_peered":"2024-03-08T08:27:39.674152+0000","last_unstale":"2024-03-08T08:27:39.677451+0000","last_undegraded":"2024-03-08T08:27:39.677451+0000","last_fullsized":"2024-03-08T08:27:39.677451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665435+0000","last_change":"2024-03-08T08:27:38.655005+0000","last_active":"2024-03-08T08:27:40.665435+0000","last_peered":"2024-03-08T08:27:40.665435+0000","last_clean":"2024-03-08T08:27:40.665435+0000","last_became_active":"2024-03-08T08:27:38.654683+0000","last_became_peered":"2024-03-08T08:27:38.654683+0000","last_unstale":"2024-03-08T08:27:40.665435+0000","last_undegraded":"2024-03-08T08:27:40.665435+0000","last_fullsized":"2024-03-08T08:27:40.665435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681765+0000","last_change":"2024-03-08T08:27:39.673959+0000","last_active":"2024-03-08T08:27:39.681765+0000","last_peered":"2024-03-08T08:27:39.681765+0000","last_clean":"2024-03-08T08:27:39.681765+0000","last_became_active":"2024-03-08T08:27:39.673369+0000","last_became_peered":"2024-03-08T08:27:39.673369+0000","last_unstale":"2024-03-08T08:27:39.681765+0000","last_undegraded":"2024-03-08T08:27:39.681765+0000","last_fullsized":"2024-03-08T08:27:39.681765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657749+0000","last_change":"2024-03-08T08:27:38.661161+0000","last_active":"2024-03-08T08:27:39.657749+0000","last_peered":"2024-03-08T08:27:39.657749+0000","last_clean":"2024-03-08T08:27:39.657749+0000","last_became_active":"2024-03-08T08:27:38.660876+0000","last_became_peered":"2024-03-08T08:27:38.660876+0000","last_unstale":"2024-03-08T08:27:39.657749+0000","last_undegraded":"2024-03-08T08:27:39.657749+0000","last_fullsized":"2024-03-08T08:27:39.657749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685847+0000","last_change":"2024-03-08T08:27:39.682454+0000","last_active":"2024-03-08T08:27:39.685847+0000","last_peered":"2024-03-08T08:27:39.685847+0000","last_clean":"2024-03-08T08:27:39.685847+0000","last_became_active":"2024-03-08T08:27:39.682178+0000","last_became_peered":"2024-03-08T08:27:39.682178+0000","last_unstale":"2024-03-08T08:27:39.685847+0000","last_undegraded":"2024-03-08T08:27:39.685847+0000","last_fullsized":"2024-03-08T08:27:39.685847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657683+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:27:39.657683+0000","last_peered":"2024-03-08T08:27:39.657683+0000","last_clean":"2024-03-08T08:27:39.657683+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:27:39.657683+0000","last_undegraded":"2024-03-08T08:27:39.657683+0000","last_fullsized":"2024-03-08T08:27:39.657683+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.668701+0000","last_change":"2024-03-08T08:27:38.659294+0000","last_active":"2024-03-08T08:27:39.668701+0000","last_peered":"2024-03-08T08:27:39.668701+0000","last_clean":"2024-03-08T08:27:39.668701+0000","last_became_active":"2024-03-08T08:27:38.659037+0000","last_became_peered":"2024-03-08T08:27:38.659037+0000","last_unstale":"2024-03-08T08:27:39.668701+0000","last_undegraded":"2024-03-08T08:27:39.668701+0000","last_fullsized":"2024-03-08T08:27:39.668701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684766+0000","last_change":"2024-03-08T08:27:39.681265+0000","last_active":"2024-03-08T08:27:39.684766+0000","last_peered":"2024-03-08T08:27:39.684766+0000","last_clean":"2024-03-08T08:27:39.684766+0000","last_became_active":"2024-03-08T08:27:39.680995+0000","last_became_peered":"2024-03-08T08:27:39.680995+0000","last_unstale":"2024-03-08T08:27:39.684766+0000","last_undegraded":"2024-03-08T08:27:39.684766+0000","last_fullsized":"2024-03-08T08:27:39.684766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657681+0000","last_change":"2024-03-08T08:27:38.663815+0000","last_active":"2024-03-08T08:27:39.657681+0000","last_peered":"2024-03-08T08:27:39.657681+0000","last_clean":"2024-03-08T08:27:39.657681+0000","last_became_active":"2024-03-08T08:27:38.663589+0000","last_became_peered":"2024-03-08T08:27:38.663589+0000","last_unstale":"2024-03-08T08:27:39.657681+0000","last_undegraded":"2024-03-08T08:27:39.657681+0000","last_fullsized":"2024-03-08T08:27:39.657681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.676559+0000","last_change":"2024-03-08T08:27:39.673596+0000","last_active":"2024-03-08T08:27:39.676559+0000","last_peered":"2024-03-08T08:27:39.676559+0000","last_clean":"2024-03-08T08:27:39.676559+0000","last_became_active":"2024-03-08T08:27:39.673171+0000","last_became_peered":"2024-03-08T08:27:39.673171+0000","last_unstale":"2024-03-08T08:27:39.676559+0000","last_undegraded":"2024-03-08T08:27:39.676559+0000","last_fullsized":"2024-03-08T08:27:39.676559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657682+0000","last_change":"2024-03-08T08:27:38.658281+0000","last_active":"2024-03-08T08:27:39.657682+0000","last_peered":"2024-03-08T08:27:39.657682+0000","last_clean":"2024-03-08T08:27:39.657682+0000","last_became_active":"2024-03-08T08:27:38.657983+0000","last_became_peered":"2024-03-08T08:27:38.657983+0000","last_unstale":"2024-03-08T08:27:39.657682+0000","last_undegraded":"2024-03-08T08:27:39.657682+0000","last_fullsized":"2024-03-08T08:27:39.657682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.662315+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.661362+0000","last_became_peered":"2024-03-08T08:27:39.661362+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.669881+0000","last_change":"2024-03-08T08:27:38.659980+0000","last_active":"2024-03-08T08:27:39.669881+0000","last_peered":"2024-03-08T08:27:39.669881+0000","last_clean":"2024-03-08T08:27:39.669881+0000","last_became_active":"2024-03-08T08:27:38.659704+0000","last_became_peered":"2024-03-08T08:27:38.659704+0000","last_unstale":"2024-03-08T08:27:39.669881+0000","last_undegraded":"2024-03-08T08:27:39.669881+0000","last_fullsized":"2024-03-08T08:27:39.669881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665238+0000","last_change":"2024-03-08T08:27:39.678619+0000","last_active":"2024-03-08T08:27:40.665238+0000","last_peered":"2024-03-08T08:27:40.665238+0000","last_clean":"2024-03-08T08:27:40.665238+0000","last_became_active":"2024-03-08T08:27:39.678371+0000","last_became_peered":"2024-03-08T08:27:39.678371+0000","last_unstale":"2024-03-08T08:27:40.665238+0000","last_undegraded":"2024-03-08T08:27:40.665238+0000","last_fullsized":"2024-03-08T08:27:40.665238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665304+0000","last_change":"2024-03-08T08:27:38.654132+0000","last_active":"2024-03-08T08:27:40.665304+0000","last_peered":"2024-03-08T08:27:40.665304+0000","last_clean":"2024-03-08T08:27:40.665304+0000","last_became_active":"2024-03-08T08:27:38.653844+0000","last_became_peered":"2024-03-08T08:27:38.653844+0000","last_unstale":"2024-03-08T08:27:40.665304+0000","last_undegraded":"2024-03-08T08:27:40.665304+0000","last_fullsized":"2024-03-08T08:27:40.665304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.680984+0000","last_change":"2024-03-08T08:27:39.679299+0000","last_active":"2024-03-08T08:27:39.680984+0000","last_peered":"2024-03-08T08:27:39.680984+0000","last_clean":"2024-03-08T08:27:39.680984+0000","last_became_active":"2024-03-08T08:27:39.679023+0000","last_became_peered":"2024-03-08T08:27:39.679023+0000","last_unstale":"2024-03-08T08:27:39.680984+0000","last_undegraded":"2024-03-08T08:27:39.680984+0000","last_fullsized":"2024-03-08T08:27:39.680984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.660086+0000","last_change":"2024-03-08T08:27:38.665656+0000","last_active":"2024-03-08T08:27:39.660086+0000","last_peered":"2024-03-08T08:27:39.660086+0000","last_clean":"2024-03-08T08:27:39.660086+0000","last_became_active":"2024-03-08T08:27:38.665370+0000","last_became_peered":"2024-03-08T08:27:38.665370+0000","last_unstale":"2024-03-08T08:27:39.660086+0000","last_undegraded":"2024-03-08T08:27:39.660086+0000","last_fullsized":"2024-03-08T08:27:39.660086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665354+0000","last_change":"2024-03-08T08:27:39.664052+0000","last_active":"2024-03-08T08:27:40.665354+0000","last_peered":"2024-03-08T08:27:40.665354+0000","last_clean":"2024-03-08T08:27:40.665354+0000","last_became_active":"2024-03-08T08:27:39.663794+0000","last_became_peered":"2024-03-08T08:27:39.663794+0000","last_unstale":"2024-03-08T08:27:40.665354+0000","last_undegraded":"2024-03-08T08:27:40.665354+0000","last_fullsized":"2024-03-08T08:27:40.665354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665496+0000","last_change":"2024-03-08T08:27:38.658214+0000","last_active":"2024-03-08T08:27:40.665496+0000","last_peered":"2024-03-08T08:27:40.665496+0000","last_clean":"2024-03-08T08:27:40.665496+0000","last_became_active":"2024-03-08T08:27:38.657944+0000","last_became_peered":"2024-03-08T08:27:38.657944+0000","last_unstale":"2024-03-08T08:27:40.665496+0000","last_undegraded":"2024-03-08T08:27:40.665496+0000","last_fullsized":"2024-03-08T08:27:40.665496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681699+0000","last_change":"2024-03-08T08:27:39.671410+0000","last_active":"2024-03-08T08:27:39.681699+0000","last_peered":"2024-03-08T08:27:39.681699+0000","last_clean":"2024-03-08T08:27:39.681699+0000","last_became_active":"2024-03-08T08:27:39.671169+0000","last_became_peered":"2024-03-08T08:27:39.671169+0000","last_unstale":"2024-03-08T08:27:39.681699+0000","last_undegraded":"2024-03-08T08:27:39.681699+0000","last_fullsized":"2024-03-08T08:27:39.681699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665140+0000","last_change":"2024-03-08T08:27:38.657833+0000","last_active":"2024-03-08T08:27:40.665140+0000","last_peered":"2024-03-08T08:27:40.665140+0000","last_clean":"2024-03-08T08:27:40.665140+0000","last_became_active":"2024-03-08T08:27:38.657560+0000","last_became_peered":"2024-03-08T08:27:38.657560+0000","last_unstale":"2024-03-08T08:27:40.665140+0000","last_undegraded":"2024-03-08T08:27:40.665140+0000","last_fullsized":"2024-03-08T08:27:40.665140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.675751+0000","last_change":"2024-03-08T08:27:39.668821+0000","last_active":"2024-03-08T08:27:39.675751+0000","last_peered":"2024-03-08T08:27:39.675751+0000","last_clean":"2024-03-08T08:27:39.675751+0000","last_became_active":"2024-03-08T08:27:39.668559+0000","last_became_peered":"2024-03-08T08:27:39.668559+0000","last_unstale":"2024-03-08T08:27:39.675751+0000","last_undegraded":"2024-03-08T08:27:39.675751+0000","last_fullsized":"2024-03-08T08:27:39.675751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:26.082729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.657614+0000","last_change":"2024-03-08T08:27:38.665148+0000","last_active":"2024-03-08T08:27:39.657614+0000","last_peered":"2024-03-08T08:27:39.657614+0000","last_clean":"2024-03-08T08:27:39.657614+0000","last_became_active":"2024-03-08T08:27:38.664834+0000","last_became_peered":"2024-03-08T08:27:38.664834+0000","last_unstale":"2024-03-08T08:27:39.657614+0000","last_undegraded":"2024-03-08T08:27:39.657614+0000","last_fullsized":"2024-03-08T08:27:39.657614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.667641+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.667296+0000","last_became_peered":"2024-03-08T08:27:39.667296+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665163+0000","last_change":"2024-03-08T08:27:38.660688+0000","last_active":"2024-03-08T08:27:40.665163+0000","last_peered":"2024-03-08T08:27:40.665163+0000","last_clean":"2024-03-08T08:27:40.665163+0000","last_became_active":"2024-03-08T08:27:38.660427+0000","last_became_peered":"2024-03-08T08:27:38.660427+0000","last_unstale":"2024-03-08T08:27:40.665163+0000","last_undegraded":"2024-03-08T08:27:40.665163+0000","last_fullsized":"2024-03-08T08:27:40.665163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.671217+0000","last_change":"2024-03-08T08:27:39.665202+0000","last_active":"2024-03-08T08:27:39.671217+0000","last_peered":"2024-03-08T08:27:39.671217+0000","last_clean":"2024-03-08T08:27:39.671217+0000","last_became_active":"2024-03-08T08:27:39.663567+0000","last_became_peered":"2024-03-08T08:27:39.663567+0000","last_unstale":"2024-03-08T08:27:39.671217+0000","last_undegraded":"2024-03-08T08:27:39.671217+0000","last_fullsized":"2024-03-08T08:27:39.671217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665293+0000","last_change":"2024-03-08T08:27:38.660223+0000","last_active":"2024-03-08T08:27:40.665293+0000","last_peered":"2024-03-08T08:27:40.665293+0000","last_clean":"2024-03-08T08:27:40.665293+0000","last_became_active":"2024-03-08T08:27:38.659939+0000","last_became_peered":"2024-03-08T08:27:38.659939+0000","last_unstale":"2024-03-08T08:27:40.665293+0000","last_undegraded":"2024-03-08T08:27:40.665293+0000","last_fullsized":"2024-03-08T08:27:40.665293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.678757+0000","last_change":"2024-03-08T08:27:39.670191+0000","last_active":"2024-03-08T08:27:39.678757+0000","last_peered":"2024-03-08T08:27:39.678757+0000","last_clean":"2024-03-08T08:27:39.678757+0000","last_became_active":"2024-03-08T08:27:39.669921+0000","last_became_peered":"2024-03-08T08:27:39.669921+0000","last_unstale":"2024-03-08T08:27:39.678757+0000","last_undegraded":"2024-03-08T08:27:39.678757+0000","last_fullsized":"2024-03-08T08:27:39.678757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685810+0000","last_change":"2024-03-08T08:27:38.665118+0000","last_active":"2024-03-08T08:27:39.685810+0000","last_peered":"2024-03-08T08:27:39.685810+0000","last_clean":"2024-03-08T08:27:39.685810+0000","last_became_active":"2024-03-08T08:27:38.664812+0000","last_became_peered":"2024-03-08T08:27:38.664812+0000","last_unstale":"2024-03-08T08:27:39.685810+0000","last_undegraded":"2024-03-08T08:27:39.685810+0000","last_fullsized":"2024-03-08T08:27:39.685810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664566+0000","last_change":"2024-03-08T08:27:39.672103+0000","last_active":"2024-03-08T08:27:40.664566+0000","last_peered":"2024-03-08T08:27:40.664566+0000","last_clean":"2024-03-08T08:27:40.664566+0000","last_became_active":"2024-03-08T08:27:39.671821+0000","last_became_peered":"2024-03-08T08:27:39.671821+0000","last_unstale":"2024-03-08T08:27:40.664566+0000","last_undegraded":"2024-03-08T08:27:40.664566+0000","last_fullsized":"2024-03-08T08:27:40.664566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.680899+0000","last_change":"2024-03-08T08:27:39.676821+0000","last_active":"2024-03-08T08:27:39.680899+0000","last_peered":"2024-03-08T08:27:39.680899+0000","last_clean":"2024-03-08T08:27:39.680899+0000","last_became_active":"2024-03-08T08:27:39.674571+0000","last_became_peered":"2024-03-08T08:27:39.674571+0000","last_unstale":"2024-03-08T08:27:39.680899+0000","last_undegraded":"2024-03-08T08:27:39.680899+0000","last_fullsized":"2024-03-08T08:27:39.680899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.686002+0000","last_change":"2024-03-08T08:27:38.667618+0000","last_active":"2024-03-08T08:27:39.686002+0000","last_peered":"2024-03-08T08:27:39.686002+0000","last_clean":"2024-03-08T08:27:39.686002+0000","last_became_active":"2024-03-08T08:27:38.667361+0000","last_became_peered":"2024-03-08T08:27:38.667361+0000","last_unstale":"2024-03-08T08:27:39.686002+0000","last_undegraded":"2024-03-08T08:27:39.686002+0000","last_fullsized":"2024-03-08T08:27:39.686002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.682985+0000","last_change":"2024-03-08T08:27:39.679501+0000","last_active":"2024-03-08T08:27:39.682985+0000","last_peered":"2024-03-08T08:27:39.682985+0000","last_clean":"2024-03-08T08:27:39.682985+0000","last_became_active":"2024-03-08T08:27:39.679204+0000","last_became_peered":"2024-03-08T08:27:39.679204+0000","last_unstale":"2024-03-08T08:27:39.682985+0000","last_undegraded":"2024-03-08T08:27:39.682985+0000","last_fullsized":"2024-03-08T08:27:39.682985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.685796+0000","last_change":"2024-03-08T08:27:38.667616+0000","last_active":"2024-03-08T08:27:39.685796+0000","last_peered":"2024-03-08T08:27:39.685796+0000","last_clean":"2024-03-08T08:27:39.685796+0000","last_became_active":"2024-03-08T08:27:38.667335+0000","last_became_peered":"2024-03-08T08:27:38.667335+0000","last_unstale":"2024-03-08T08:27:39.685796+0000","last_undegraded":"2024-03-08T08:27:39.685796+0000","last_fullsized":"2024-03-08T08:27:39.685796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.684517+0000","last_change":"2024-03-08T08:27:39.681042+0000","last_active":"2024-03-08T08:27:39.684517+0000","last_peered":"2024-03-08T08:27:39.684517+0000","last_clean":"2024-03-08T08:27:39.684517+0000","last_became_active":"2024-03-08T08:27:39.680791+0000","last_became_peered":"2024-03-08T08:27:39.680791+0000","last_unstale":"2024-03-08T08:27:39.684517+0000","last_undegraded":"2024-03-08T08:27:39.684517+0000","last_fullsized":"2024-03-08T08:27:39.684517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.681792+0000","last_change":"2024-03-08T08:27:38.659145+0000","last_active":"2024-03-08T08:27:39.681792+0000","last_peered":"2024-03-08T08:27:39.681792+0000","last_clean":"2024-03-08T08:27:39.681792+0000","last_became_active":"2024-03-08T08:27:38.658848+0000","last_became_peered":"2024-03-08T08:27:38.658848+0000","last_unstale":"2024-03-08T08:27:39.681792+0000","last_undegraded":"2024-03-08T08:27:39.681792+0000","last_fullsized":"2024-03-08T08:27:39.681792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.678663+0000","last_change":"2024-03-08T08:27:39.668041+0000","last_active":"2024-03-08T08:27:39.678663+0000","last_peered":"2024-03-08T08:27:39.678663+0000","last_clean":"2024-03-08T08:27:39.678663+0000","last_became_active":"2024-03-08T08:27:39.667724+0000","last_became_peered":"2024-03-08T08:27:39.667724+0000","last_unstale":"2024-03-08T08:27:39.678663+0000","last_undegraded":"2024-03-08T08:27:39.678663+0000","last_fullsized":"2024-03-08T08:27:39.678663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-08T08:27:39.686112+0000","last_change":"2024-03-08T08:27:38.657913+0000","last_active":"2024-03-08T08:27:39.686112+0000","last_peered":"2024-03-08T08:27:39.686112+0000","last_clean":"2024-03-08T08:27:39.686112+0000","last_became_active":"2024-03-08T08:27:38.657593+0000","last_became_peered":"2024-03-08T08:27:38.657593+0000","last_unstale":"2024-03-08T08:27:39.686112+0000","last_undegraded":"2024-03-08T08:27:39.686112+0000","last_fullsized":"2024-03-08T08:27:39.686112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:27:46.999 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:27:47.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.107+0000 7fe53befb640 1 Processor -- start 2024-03-08T08:27:47.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe53befb640 1 -- start start 2024-03-08T08:27:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe53befb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe53415e340 0x7fe53406b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe53befb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534066990 0x7fe5340d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe53befb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53415e720 con 0x7fe5340d05f0 2024-03-08T08:27:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe53befb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53415f020 con 0x7fe53415e340 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe53befb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe534066ec0 con 0x7fe534066990 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe539c97640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe53415e340 0x7fe53406b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe539c97640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe53415e340 0x7fe53406b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59200/0 (socket says 172.21.15.105:59200) 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.108+0000 7fe539c97640 1 -- 172.21.15.105:0/3270677458 learned_addr learned my addr 172.21.15.105:0/3270677458 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.109+0000 7fe539496640 1 --2- 172.21.15.105:0/3270677458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534066990 0x7fe5340d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.109+0000 7fe539c97640 1 -- 172.21.15.105:0/3270677458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534066990 msgr2=0x7fe5340d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.109+0000 7fe539c97640 1 --2- 172.21.15.105:0/3270677458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534066990 0x7fe5340d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.109+0000 7fe539c97640 1 -- 172.21.15.105:0/3270677458 >> v1:172.21.15.144:6789/0 conn(0x7fe5340d05f0 legacy=0x7fe534161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.109+0000 7fe539c97640 1 -- 172.21.15.105:0/3270677458 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53415e9d0 con 0x7fe53415e340 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.109+0000 7fe539c97640 1 --2- 172.21.15.105:0/3270677458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe53415e340 0x7fe53406b690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe530009e80 tx=0x7fe53002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0ce2c03dc3d5a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe538c95640 1 -- 172.21.15.105:0/3270677458 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53002c930 con 0x7fe53415e340 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe538c95640 1 -- 172.21.15.105:0/3270677458 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53002cad0 con 0x7fe53415e340 2024-03-08T08:27:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe538c95640 1 -- 172.21.15.105:0/3270677458 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300343f0 con 0x7fe53415e340 2024-03-08T08:27:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe53befb640 1 -- 172.21.15.105:0/3270677458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe53415e340 msgr2=0x7fe53406b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe53befb640 1 --2- 172.21.15.105:0/3270677458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe53415e340 0x7fe53406b690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe530009e80 tx=0x7fe53002bd90 comp rx=0 tx=0).stop 2024-03-08T08:27:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe53befb640 1 -- 172.21.15.105:0/3270677458 shutdown_connections 2024-03-08T08:27:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe53befb640 1 --2- 172.21.15.105:0/3270677458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe534066990 0x7fe5340d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe53befb640 1 --2- 172.21.15.105:0/3270677458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe53415e340 0x7fe53406b690 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe53befb640 1 -- 172.21.15.105:0/3270677458 >> 172.21.15.105:0/3270677458 conn(0x7fe534072110 msgr2=0x7fe534072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe53befb640 1 -- 172.21.15.105:0/3270677458 shutdown_connections 2024-03-08T08:27:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.110+0000 7fe53befb640 1 -- 172.21.15.105:0/3270677458 wait complete. 2024-03-08T08:27:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe53befb640 1 Processor -- start 2024-03-08T08:27:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe53befb640 1 -- start start 2024-03-08T08:27:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe53befb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe534066990 0x7fe5340998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe53befb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5340d05f0 0x7fe53414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe53befb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415e340 0x7fe53414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe53befb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415f4b0 con 0x7fe534066990 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe53befb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340af530 con 0x7fe53415e340 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe53befb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340b0f00 con 0x7fe5340d05f0 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe539c97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe534066990 0x7fe5340998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe539c97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe534066990 0x7fe5340998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59212/0 (socket says 172.21.15.105:59212) 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.112+0000 7fe539c97640 1 -- 172.21.15.105:0/2965370817 learned_addr learned my addr 172.21.15.105:0/2965370817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe539496640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5340d05f0 0x7fe53414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe539c97640 1 -- 172.21.15.105:0/2965370817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5340d05f0 msgr2=0x7fe53414dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe539c97640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5340d05f0 0x7fe53414dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe539c97640 1 -- 172.21.15.105:0/2965370817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415e340 msgr2=0x7fe53414e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe53a498640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415e340 0x7fe53414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe539c97640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415e340 0x7fe53414e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe539c97640 1 -- 172.21.15.105:0/2965370817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53414e760 con 0x7fe534066990 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe539c97640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe534066990 0x7fe5340998c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe530009e80 tx=0x7fe530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe52a7fc640 1 -- 172.21.15.105:0/2965370817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530043020 con 0x7fe534066990 2024-03-08T08:27:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe52a7fc640 1 -- 172.21.15.105:0/2965370817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53003e450 con 0x7fe534066990 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53414ea40 con 0x7fe534066990 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe52a7fc640 1 -- 172.21.15.105:0/2965370817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53003e840 con 0x7fe534066990 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.113+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5340b0b90 con 0x7fe534066990 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.114+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe534075120 con 0x7fe534066990 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.114+0000 7fe52a7fc640 1 -- 172.21.15.105:0/2965370817 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe530046020 con 0x7fe534066990 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.115+0000 7fe52a7fc640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe51806b500 0x7fe51806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.115+0000 7fe52a7fc640 1 -- 172.21.15.105:0/2965370817 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5300ac7c0 con 0x7fe534066990 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.115+0000 7fe539496640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe51806b500 0x7fe51806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.115+0000 7fe539496640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe51806b500 0x7fe51806d9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.119+0000 7fe52a7fc640 1 -- 172.21.15.105:0/2965370817 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe53004d070 con 0x7fe534066990 2024-03-08T08:27:47.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.287+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5340ad0d0 con 0x7fe51806b500 2024-03-08T08:27:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.293+0000 7fe52a7fc640 1 -- 172.21.15.105:0/2965370817 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251860 (secure 0 0 0) 0x7fe5340ad0d0 con 0x7fe51806b500 2024-03-08T08:27:47.298 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:47.299 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:27:47.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe51806b500 msgr2=0x7fe51806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe51806b500 0x7fe51806d9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).stop 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe534066990 msgr2=0x7fe5340998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe534066990 0x7fe5340998c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe530009e80 tx=0x7fe530004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 shutdown_connections 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe51806b500 0x7fe51806d9c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe53415e340 0x7fe53414e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5340d05f0 0x7fe53414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 --2- 172.21.15.105:0/2965370817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe534066990 0x7fe5340998c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 >> 172.21.15.105:0/2965370817 conn(0x7fe534072110 msgr2=0x7fe5340ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 shutdown_connections 2024-03-08T08:27:47.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.296+0000 7fe53befb640 1 -- 172.21.15.105:0/2965370817 wait complete. 2024-03-08T08:27:47.308 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-08T08:27:47.097235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001615"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285490+0000","last_change":"2024-03-08T08:27:39.669170+0000","last_active":"2024-03-08T08:27:42.285490+0000","last_peered":"2024-03-08T08:27:42.285490+0000","last_clean":"2024-03-08T08:27:42.285490+0000","last_became_active":"2024-03-08T08:27:39.668776+0000","last_became_peered":"2024-03-08T08:27:39.668776+0000","last_unstale":"2024-03-08T08:27:42.285490+0000","last_undegraded":"2024-03-08T08:27:42.285490+0000","last_fullsized":"2024-03-08T08:27:42.285490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:58.646243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665542+0000","last_change":"2024-03-08T08:27:39.669937+0000","last_active":"2024-03-08T08:27:40.665542+0000","last_peered":"2024-03-08T08:27:40.665542+0000","last_clean":"2024-03-08T08:27:40.665542+0000","last_became_active":"2024-03-08T08:27:39.669643+0000","last_became_peered":"2024-03-08T08:27:39.669643+0000","last_unstale":"2024-03-08T08:27:40.665542+0000","last_undegraded":"2024-03-08T08:27:40.665542+0000","last_fullsized":"2024-03-08T08:27:40.665542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229937+0000","last_change":"2024-03-08T08:27:39.669698+0000","last_active":"2024-03-08T08:27:43.229937+0000","last_peered":"2024-03-08T08:27:43.229937+0000","last_clean":"2024-03-08T08:27:43.229937+0000","last_became_active":"2024-03-08T08:27:39.669412+0000","last_became_peered":"2024-03-08T08:27:39.669412+0000","last_unstale":"2024-03-08T08:27:43.229937+0000","last_undegraded":"2024-03-08T08:27:43.229937+0000","last_fullsized":"2024-03-08T08:27:43.229937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229311+0000","last_change":"2024-03-08T08:27:39.674216+0000","last_active":"2024-03-08T08:27:43.229311+0000","last_peered":"2024-03-08T08:27:43.229311+0000","last_clean":"2024-03-08T08:27:43.229311+0000","last_became_active":"2024-03-08T08:27:39.673664+0000","last_became_peered":"2024-03-08T08:27:39.673664+0000","last_unstale":"2024-03-08T08:27:43.229311+0000","last_undegraded":"2024-03-08T08:27:43.229311+0000","last_fullsized":"2024-03-08T08:27:43.229311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285273+0000","last_change":"2024-03-08T08:27:39.671956+0000","last_active":"2024-03-08T08:27:42.285273+0000","last_peered":"2024-03-08T08:27:42.285273+0000","last_clean":"2024-03-08T08:27:42.285273+0000","last_became_active":"2024-03-08T08:27:39.671671+0000","last_became_peered":"2024-03-08T08:27:39.671671+0000","last_unstale":"2024-03-08T08:27:42.285273+0000","last_undegraded":"2024-03-08T08:27:42.285273+0000","last_fullsized":"2024-03-08T08:27:42.285273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229106+0000","last_change":"2024-03-08T08:27:39.681569+0000","last_active":"2024-03-08T08:27:43.229106+0000","last_peered":"2024-03-08T08:27:43.229106+0000","last_clean":"2024-03-08T08:27:43.229106+0000","last_became_active":"2024-03-08T08:27:39.681306+0000","last_became_peered":"2024-03-08T08:27:39.681306+0000","last_unstale":"2024-03-08T08:27:43.229106+0000","last_undegraded":"2024-03-08T08:27:43.229106+0000","last_fullsized":"2024-03-08T08:27:43.229106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285103+0000","last_change":"2024-03-08T08:27:39.669471+0000","last_active":"2024-03-08T08:27:42.285103+0000","last_peered":"2024-03-08T08:27:42.285103+0000","last_clean":"2024-03-08T08:27:42.285103+0000","last_became_active":"2024-03-08T08:27:39.669144+0000","last_became_peered":"2024-03-08T08:27:39.669144+0000","last_unstale":"2024-03-08T08:27:42.285103+0000","last_undegraded":"2024-03-08T08:27:42.285103+0000","last_fullsized":"2024-03-08T08:27:42.285103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228726+0000","last_change":"2024-03-08T08:27:39.680587+0000","last_active":"2024-03-08T08:27:43.228726+0000","last_peered":"2024-03-08T08:27:43.228726+0000","last_clean":"2024-03-08T08:27:43.228726+0000","last_became_active":"2024-03-08T08:27:39.680354+0000","last_became_peered":"2024-03-08T08:27:39.680354+0000","last_unstale":"2024-03-08T08:27:43.228726+0000","last_undegraded":"2024-03-08T08:27:43.228726+0000","last_fullsized":"2024-03-08T08:27:43.228726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285685+0000","last_change":"2024-03-08T08:27:39.682744+0000","last_active":"2024-03-08T08:27:42.285685+0000","last_peered":"2024-03-08T08:27:42.285685+0000","last_clean":"2024-03-08T08:27:42.285685+0000","last_became_active":"2024-03-08T08:27:39.682486+0000","last_became_peered":"2024-03-08T08:27:39.682486+0000","last_unstale":"2024-03-08T08:27:42.285685+0000","last_undegraded":"2024-03-08T08:27:42.285685+0000","last_fullsized":"2024-03-08T08:27:42.285685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:01:26.479387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665493+0000","last_change":"2024-03-08T08:27:39.681720+0000","last_active":"2024-03-08T08:27:40.665493+0000","last_peered":"2024-03-08T08:27:40.665493+0000","last_clean":"2024-03-08T08:27:40.665493+0000","last_became_active":"2024-03-08T08:27:39.681473+0000","last_became_peered":"2024-03-08T08:27:39.681473+0000","last_unstale":"2024-03-08T08:27:40.665493+0000","last_undegraded":"2024-03-08T08:27:40.665493+0000","last_fullsized":"2024-03-08T08:27:40.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.681092+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.680824+0000","last_became_peered":"2024-03-08T08:27:39.680824+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665505+0000","last_change":"2024-03-08T08:27:39.674803+0000","last_active":"2024-03-08T08:27:40.665505+0000","last_peered":"2024-03-08T08:27:40.665505+0000","last_clean":"2024-03-08T08:27:40.665505+0000","last_became_active":"2024-03-08T08:27:39.674320+0000","last_became_peered":"2024-03-08T08:27:39.674320+0000","last_unstale":"2024-03-08T08:27:40.665505+0000","last_undegraded":"2024-03-08T08:27:40.665505+0000","last_fullsized":"2024-03-08T08:27:40.665505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228821+0000","last_change":"2024-03-08T08:27:39.665926+0000","last_active":"2024-03-08T08:27:43.228821+0000","last_peered":"2024-03-08T08:27:43.228821+0000","last_clean":"2024-03-08T08:27:43.228821+0000","last_became_active":"2024-03-08T08:27:39.665607+0000","last_became_peered":"2024-03-08T08:27:39.665607+0000","last_unstale":"2024-03-08T08:27:43.228821+0000","last_undegraded":"2024-03-08T08:27:43.228821+0000","last_fullsized":"2024-03-08T08:27:43.228821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229697+0000","last_change":"2024-03-08T08:27:39.666842+0000","last_active":"2024-03-08T08:27:43.229697+0000","last_peered":"2024-03-08T08:27:43.229697+0000","last_clean":"2024-03-08T08:27:43.229697+0000","last_became_active":"2024-03-08T08:27:39.666519+0000","last_became_peered":"2024-03-08T08:27:39.666519+0000","last_unstale":"2024-03-08T08:27:43.229697+0000","last_undegraded":"2024-03-08T08:27:43.229697+0000","last_fullsized":"2024-03-08T08:27:43.229697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665072+0000","last_change":"2024-03-08T08:27:39.672195+0000","last_active":"2024-03-08T08:27:40.665072+0000","last_peered":"2024-03-08T08:27:40.665072+0000","last_clean":"2024-03-08T08:27:40.665072+0000","last_became_active":"2024-03-08T08:27:39.671927+0000","last_became_peered":"2024-03-08T08:27:39.671927+0000","last_unstale":"2024-03-08T08:27:40.665072+0000","last_undegraded":"2024-03-08T08:27:40.665072+0000","last_fullsized":"2024-03-08T08:27:40.665072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229048+0000","last_change":"2024-03-08T08:27:39.677932+0000","last_active":"2024-03-08T08:27:43.229048+0000","last_peered":"2024-03-08T08:27:43.229048+0000","last_clean":"2024-03-08T08:27:43.229048+0000","last_became_active":"2024-03-08T08:27:39.674418+0000","last_became_peered":"2024-03-08T08:27:39.674418+0000","last_unstale":"2024-03-08T08:27:43.229048+0000","last_undegraded":"2024-03-08T08:27:43.229048+0000","last_fullsized":"2024-03-08T08:27:43.229048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229734+0000","last_change":"2024-03-08T08:27:39.680336+0000","last_active":"2024-03-08T08:27:43.229734+0000","last_peered":"2024-03-08T08:27:43.229734+0000","last_clean":"2024-03-08T08:27:43.229734+0000","last_became_active":"2024-03-08T08:27:39.680072+0000","last_became_peered":"2024-03-08T08:27:39.680072+0000","last_unstale":"2024-03-08T08:27:43.229734+0000","last_undegraded":"2024-03-08T08:27:43.229734+0000","last_fullsized":"2024-03-08T08:27:43.229734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:05.572283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228863+0000","last_change":"2024-03-08T08:27:39.663330+0000","last_active":"2024-03-08T08:27:43.228863+0000","last_peered":"2024-03-08T08:27:43.228863+0000","last_clean":"2024-03-08T08:27:43.228863+0000","last_became_active":"2024-03-08T08:27:39.663063+0000","last_became_peered":"2024-03-08T08:27:39.663063+0000","last_unstale":"2024-03-08T08:27:43.228863+0000","last_undegraded":"2024-03-08T08:27:43.228863+0000","last_fullsized":"2024-03-08T08:27:43.228863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285630+0000","last_change":"2024-03-08T08:27:39.673627+0000","last_active":"2024-03-08T08:27:42.285630+0000","last_peered":"2024-03-08T08:27:42.285630+0000","last_clean":"2024-03-08T08:27:42.285630+0000","last_became_active":"2024-03-08T08:27:39.673186+0000","last_became_peered":"2024-03-08T08:27:39.673186+0000","last_unstale":"2024-03-08T08:27:42.285630+0000","last_undegraded":"2024-03-08T08:27:42.285630+0000","last_fullsized":"2024-03-08T08:27:42.285630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:05.572283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664828+0000","last_change":"2024-03-08T08:27:39.672747+0000","last_active":"2024-03-08T08:27:40.664828+0000","last_peered":"2024-03-08T08:27:40.664828+0000","last_clean":"2024-03-08T08:27:40.664828+0000","last_became_active":"2024-03-08T08:27:39.672478+0000","last_became_peered":"2024-03-08T08:27:39.672478+0000","last_unstale":"2024-03-08T08:27:40.664828+0000","last_undegraded":"2024-03-08T08:27:40.664828+0000","last_fullsized":"2024-03-08T08:27:40.664828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229825+0000","last_change":"2024-03-08T08:27:39.671025+0000","last_active":"2024-03-08T08:27:43.229825+0000","last_peered":"2024-03-08T08:27:43.229825+0000","last_clean":"2024-03-08T08:27:43.229825+0000","last_became_active":"2024-03-08T08:27:39.670767+0000","last_became_peered":"2024-03-08T08:27:39.670767+0000","last_unstale":"2024-03-08T08:27:43.229825+0000","last_undegraded":"2024-03-08T08:27:43.229825+0000","last_fullsized":"2024-03-08T08:27:43.229825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229139+0000","last_change":"2024-03-08T08:27:39.680602+0000","last_active":"2024-03-08T08:27:43.229139+0000","last_peered":"2024-03-08T08:27:43.229139+0000","last_clean":"2024-03-08T08:27:43.229139+0000","last_became_active":"2024-03-08T08:27:39.679691+0000","last_became_peered":"2024-03-08T08:27:39.679691+0000","last_unstale":"2024-03-08T08:27:43.229139+0000","last_undegraded":"2024-03-08T08:27:43.229139+0000","last_fullsized":"2024-03-08T08:27:43.229139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229185+0000","last_change":"2024-03-08T08:27:39.673063+0000","last_active":"2024-03-08T08:27:43.229185+0000","last_peered":"2024-03-08T08:27:43.229185+0000","last_clean":"2024-03-08T08:27:43.229185+0000","last_became_active":"2024-03-08T08:27:39.672801+0000","last_became_peered":"2024-03-08T08:27:39.672801+0000","last_unstale":"2024-03-08T08:27:43.229185+0000","last_undegraded":"2024-03-08T08:27:43.229185+0000","last_fullsized":"2024-03-08T08:27:43.229185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229108+0000","last_change":"2024-03-08T08:27:39.680324+0000","last_active":"2024-03-08T08:27:43.229108+0000","last_peered":"2024-03-08T08:27:43.229108+0000","last_clean":"2024-03-08T08:27:43.229108+0000","last_became_active":"2024-03-08T08:27:39.679988+0000","last_became_peered":"2024-03-08T08:27:39.679988+0000","last_unstale":"2024-03-08T08:27:43.229108+0000","last_undegraded":"2024-03-08T08:27:43.229108+0000","last_fullsized":"2024-03-08T08:27:43.229108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228956+0000","last_change":"2024-03-08T08:27:39.669492+0000","last_active":"2024-03-08T08:27:43.228956+0000","last_peered":"2024-03-08T08:27:43.228956+0000","last_clean":"2024-03-08T08:27:43.228956+0000","last_became_active":"2024-03-08T08:27:39.669223+0000","last_became_peered":"2024-03-08T08:27:39.669223+0000","last_unstale":"2024-03-08T08:27:43.228956+0000","last_undegraded":"2024-03-08T08:27:43.228956+0000","last_fullsized":"2024-03-08T08:27:43.228956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665530+0000","last_change":"2024-03-08T08:27:39.675074+0000","last_active":"2024-03-08T08:27:40.665530+0000","last_peered":"2024-03-08T08:27:40.665530+0000","last_clean":"2024-03-08T08:27:40.665530+0000","last_became_active":"2024-03-08T08:27:39.674811+0000","last_became_peered":"2024-03-08T08:27:39.674811+0000","last_unstale":"2024-03-08T08:27:40.665530+0000","last_undegraded":"2024-03-08T08:27:40.665530+0000","last_fullsized":"2024-03-08T08:27:40.665530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665374+0000","last_change":"2024-03-08T08:27:39.666768+0000","last_active":"2024-03-08T08:27:40.665374+0000","last_peered":"2024-03-08T08:27:40.665374+0000","last_clean":"2024-03-08T08:27:40.665374+0000","last_became_active":"2024-03-08T08:27:39.666516+0000","last_became_peered":"2024-03-08T08:27:39.666516+0000","last_unstale":"2024-03-08T08:27:40.665374+0000","last_undegraded":"2024-03-08T08:27:40.665374+0000","last_fullsized":"2024-03-08T08:27:40.665374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229242+0000","last_change":"2024-03-08T08:27:39.672450+0000","last_active":"2024-03-08T08:27:43.229242+0000","last_peered":"2024-03-08T08:27:43.229242+0000","last_clean":"2024-03-08T08:27:43.229242+0000","last_became_active":"2024-03-08T08:27:39.671942+0000","last_became_peered":"2024-03-08T08:27:39.671942+0000","last_unstale":"2024-03-08T08:27:43.229242+0000","last_undegraded":"2024-03-08T08:27:43.229242+0000","last_fullsized":"2024-03-08T08:27:43.229242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229974+0000","last_change":"2024-03-08T08:27:39.668257+0000","last_active":"2024-03-08T08:27:43.229974+0000","last_peered":"2024-03-08T08:27:43.229974+0000","last_clean":"2024-03-08T08:27:43.229974+0000","last_became_active":"2024-03-08T08:27:39.667225+0000","last_became_peered":"2024-03-08T08:27:39.667225+0000","last_unstale":"2024-03-08T08:27:43.229974+0000","last_undegraded":"2024-03-08T08:27:43.229974+0000","last_fullsized":"2024-03-08T08:27:43.229974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229321+0000","last_change":"2024-03-08T08:27:39.680846+0000","last_active":"2024-03-08T08:27:43.229321+0000","last_peered":"2024-03-08T08:27:43.229321+0000","last_clean":"2024-03-08T08:27:43.229321+0000","last_became_active":"2024-03-08T08:27:39.680341+0000","last_became_peered":"2024-03-08T08:27:39.680341+0000","last_unstale":"2024-03-08T08:27:43.229321+0000","last_undegraded":"2024-03-08T08:27:43.229321+0000","last_fullsized":"2024-03-08T08:27:43.229321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229315+0000","last_change":"2024-03-08T08:27:39.679406+0000","last_active":"2024-03-08T08:27:43.229315+0000","last_peered":"2024-03-08T08:27:43.229315+0000","last_clean":"2024-03-08T08:27:43.229315+0000","last_became_active":"2024-03-08T08:27:39.679148+0000","last_became_peered":"2024-03-08T08:27:39.679148+0000","last_unstale":"2024-03-08T08:27:43.229315+0000","last_undegraded":"2024-03-08T08:27:43.229315+0000","last_fullsized":"2024-03-08T08:27:43.229315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664398+0000","last_change":"2024-03-08T08:27:39.670032+0000","last_active":"2024-03-08T08:27:40.664398+0000","last_peered":"2024-03-08T08:27:40.664398+0000","last_clean":"2024-03-08T08:27:40.664398+0000","last_became_active":"2024-03-08T08:27:39.669704+0000","last_became_peered":"2024-03-08T08:27:39.669704+0000","last_unstale":"2024-03-08T08:27:40.664398+0000","last_undegraded":"2024-03-08T08:27:40.664398+0000","last_fullsized":"2024-03-08T08:27:40.664398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665149+0000","last_change":"2024-03-08T08:27:39.671209+0000","last_active":"2024-03-08T08:27:40.665149+0000","last_peered":"2024-03-08T08:27:40.665149+0000","last_clean":"2024-03-08T08:27:40.665149+0000","last_became_active":"2024-03-08T08:27:39.670900+0000","last_became_peered":"2024-03-08T08:27:39.670900+0000","last_unstale":"2024-03-08T08:27:40.665149+0000","last_undegraded":"2024-03-08T08:27:40.665149+0000","last_fullsized":"2024-03-08T08:27:40.665149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229061+0000","last_change":"2024-03-08T08:27:38.661099+0000","last_active":"2024-03-08T08:27:43.229061+0000","last_peered":"2024-03-08T08:27:43.229061+0000","last_clean":"2024-03-08T08:27:43.229061+0000","last_became_active":"2024-03-08T08:27:38.660829+0000","last_became_peered":"2024-03-08T08:27:38.660829+0000","last_unstale":"2024-03-08T08:27:43.229061+0000","last_undegraded":"2024-03-08T08:27:43.229061+0000","last_fullsized":"2024-03-08T08:27:43.229061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230116+0000","last_change":"2024-03-08T08:27:39.671575+0000","last_active":"2024-03-08T08:27:43.230116+0000","last_peered":"2024-03-08T08:27:43.230116+0000","last_clean":"2024-03-08T08:27:43.230116+0000","last_became_active":"2024-03-08T08:27:39.671311+0000","last_became_peered":"2024-03-08T08:27:39.671311+0000","last_unstale":"2024-03-08T08:27:43.230116+0000","last_undegraded":"2024-03-08T08:27:43.230116+0000","last_fullsized":"2024-03-08T08:27:43.230116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664688+0000","last_change":"2024-03-08T08:27:38.651812+0000","last_active":"2024-03-08T08:27:40.664688+0000","last_peered":"2024-03-08T08:27:40.664688+0000","last_clean":"2024-03-08T08:27:40.664688+0000","last_became_active":"2024-03-08T08:27:38.651040+0000","last_became_peered":"2024-03-08T08:27:38.651040+0000","last_unstale":"2024-03-08T08:27:40.664688+0000","last_undegraded":"2024-03-08T08:27:40.664688+0000","last_fullsized":"2024-03-08T08:27:40.664688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229327+0000","last_change":"2024-03-08T08:27:38.663033+0000","last_active":"2024-03-08T08:27:43.229327+0000","last_peered":"2024-03-08T08:27:43.229327+0000","last_clean":"2024-03-08T08:27:43.229327+0000","last_became_active":"2024-03-08T08:27:38.662753+0000","last_became_peered":"2024-03-08T08:27:38.662753+0000","last_unstale":"2024-03-08T08:27:43.229327+0000","last_undegraded":"2024-03-08T08:27:43.229327+0000","last_fullsized":"2024-03-08T08:27:43.229327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:13:30.999947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228732+0000","last_change":"2024-03-08T08:27:39.670839+0000","last_active":"2024-03-08T08:27:43.228732+0000","last_peered":"2024-03-08T08:27:43.228732+0000","last_clean":"2024-03-08T08:27:43.228732+0000","last_became_active":"2024-03-08T08:27:39.670559+0000","last_became_peered":"2024-03-08T08:27:39.670559+0000","last_unstale":"2024-03-08T08:27:43.228732+0000","last_undegraded":"2024-03-08T08:27:43.228732+0000","last_fullsized":"2024-03-08T08:27:43.228732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229150+0000","last_change":"2024-03-08T08:27:38.656237+0000","last_active":"2024-03-08T08:27:43.229150+0000","last_peered":"2024-03-08T08:27:43.229150+0000","last_clean":"2024-03-08T08:27:43.229150+0000","last_became_active":"2024-03-08T08:27:38.655944+0000","last_became_peered":"2024-03-08T08:27:38.655944+0000","last_unstale":"2024-03-08T08:27:43.229150+0000","last_undegraded":"2024-03-08T08:27:43.229150+0000","last_fullsized":"2024-03-08T08:27:43.229150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.667881+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.666906+0000","last_became_peered":"2024-03-08T08:27:39.666906+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665029+0000","last_change":"2024-03-08T08:27:38.660261+0000","last_active":"2024-03-08T08:27:40.665029+0000","last_peered":"2024-03-08T08:27:40.665029+0000","last_clean":"2024-03-08T08:27:40.665029+0000","last_became_active":"2024-03-08T08:27:38.659979+0000","last_became_peered":"2024-03-08T08:27:38.659979+0000","last_unstale":"2024-03-08T08:27:40.665029+0000","last_undegraded":"2024-03-08T08:27:40.665029+0000","last_fullsized":"2024-03-08T08:27:40.665029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229622+0000","last_change":"2024-03-08T08:27:39.679621+0000","last_active":"2024-03-08T08:27:43.229622+0000","last_peered":"2024-03-08T08:27:43.229622+0000","last_clean":"2024-03-08T08:27:43.229622+0000","last_became_active":"2024-03-08T08:27:39.679327+0000","last_became_peered":"2024-03-08T08:27:39.679327+0000","last_unstale":"2024-03-08T08:27:43.229622+0000","last_undegraded":"2024-03-08T08:27:43.229622+0000","last_fullsized":"2024-03-08T08:27:43.229622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:07.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229731+0000","last_change":"2024-03-08T08:27:38.662435+0000","last_active":"2024-03-08T08:27:43.229731+0000","last_peered":"2024-03-08T08:27:43.229731+0000","last_clean":"2024-03-08T08:27:43.229731+0000","last_became_active":"2024-03-08T08:27:38.662189+0000","last_became_peered":"2024-03-08T08:27:38.662189+0000","last_unstale":"2024-03-08T08:27:43.229731+0000","last_undegraded":"2024-03-08T08:27:43.229731+0000","last_fullsized":"2024-03-08T08:27:43.229731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230094+0000","last_change":"2024-03-08T08:27:39.680354+0000","last_active":"2024-03-08T08:27:43.230094+0000","last_peered":"2024-03-08T08:27:43.230094+0000","last_clean":"2024-03-08T08:27:43.230094+0000","last_became_active":"2024-03-08T08:27:39.680068+0000","last_became_peered":"2024-03-08T08:27:39.680068+0000","last_unstale":"2024-03-08T08:27:43.230094+0000","last_undegraded":"2024-03-08T08:27:43.230094+0000","last_fullsized":"2024-03-08T08:27:43.230094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665344+0000","last_change":"2024-03-08T08:27:38.662811+0000","last_active":"2024-03-08T08:27:40.665344+0000","last_peered":"2024-03-08T08:27:40.665344+0000","last_clean":"2024-03-08T08:27:40.665344+0000","last_became_active":"2024-03-08T08:27:38.662484+0000","last_became_peered":"2024-03-08T08:27:38.662484+0000","last_unstale":"2024-03-08T08:27:40.665344+0000","last_undegraded":"2024-03-08T08:27:40.665344+0000","last_fullsized":"2024-03-08T08:27:40.665344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665494+0000","last_change":"2024-03-08T08:27:39.681653+0000","last_active":"2024-03-08T08:27:40.665494+0000","last_peered":"2024-03-08T08:27:40.665494+0000","last_clean":"2024-03-08T08:27:40.665494+0000","last_became_active":"2024-03-08T08:27:39.681368+0000","last_became_peered":"2024-03-08T08:27:39.681368+0000","last_unstale":"2024-03-08T08:27:40.665494+0000","last_undegraded":"2024-03-08T08:27:40.665494+0000","last_fullsized":"2024-03-08T08:27:40.665494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664972+0000","last_change":"2024-03-08T08:27:38.655367+0000","last_active":"2024-03-08T08:27:40.664972+0000","last_peered":"2024-03-08T08:27:40.664972+0000","last_clean":"2024-03-08T08:27:40.664972+0000","last_became_active":"2024-03-08T08:27:38.655051+0000","last_became_peered":"2024-03-08T08:27:38.655051+0000","last_unstale":"2024-03-08T08:27:40.664972+0000","last_undegraded":"2024-03-08T08:27:40.664972+0000","last_fullsized":"2024-03-08T08:27:40.664972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285287+0000","last_change":"2024-03-08T08:27:39.672887+0000","last_active":"2024-03-08T08:27:42.285287+0000","last_peered":"2024-03-08T08:27:42.285287+0000","last_clean":"2024-03-08T08:27:42.285287+0000","last_became_active":"2024-03-08T08:27:39.672009+0000","last_became_peered":"2024-03-08T08:27:39.672009+0000","last_unstale":"2024-03-08T08:27:42.285287+0000","last_undegraded":"2024-03-08T08:27:42.285287+0000","last_fullsized":"2024-03-08T08:27:42.285287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230150+0000","last_change":"2024-03-08T08:27:38.650529+0000","last_active":"2024-03-08T08:27:43.230150+0000","last_peered":"2024-03-08T08:27:43.230150+0000","last_clean":"2024-03-08T08:27:43.230150+0000","last_became_active":"2024-03-08T08:27:38.650183+0000","last_became_peered":"2024-03-08T08:27:38.650183+0000","last_unstale":"2024-03-08T08:27:43.230150+0000","last_undegraded":"2024-03-08T08:27:43.230150+0000","last_fullsized":"2024-03-08T08:27:43.230150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:18:22.093931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229702+0000","last_change":"2024-03-08T08:27:39.670409+0000","last_active":"2024-03-08T08:27:43.229702+0000","last_peered":"2024-03-08T08:27:43.229702+0000","last_clean":"2024-03-08T08:27:43.229702+0000","last_became_active":"2024-03-08T08:27:39.670105+0000","last_became_peered":"2024-03-08T08:27:39.670105+0000","last_unstale":"2024-03-08T08:27:43.229702+0000","last_undegraded":"2024-03-08T08:27:43.229702+0000","last_fullsized":"2024-03-08T08:27:43.229702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229241+0000","last_change":"2024-03-08T08:27:38.660419+0000","last_active":"2024-03-08T08:27:43.229241+0000","last_peered":"2024-03-08T08:27:43.229241+0000","last_clean":"2024-03-08T08:27:43.229241+0000","last_became_active":"2024-03-08T08:27:38.660225+0000","last_became_peered":"2024-03-08T08:27:38.660225+0000","last_unstale":"2024-03-08T08:27:43.229241+0000","last_undegraded":"2024-03-08T08:27:43.229241+0000","last_fullsized":"2024-03-08T08:27:43.229241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285354+0000","last_change":"2024-03-08T08:27:39.675352+0000","last_active":"2024-03-08T08:27:42.285354+0000","last_peered":"2024-03-08T08:27:42.285354+0000","last_clean":"2024-03-08T08:27:42.285354+0000","last_became_active":"2024-03-08T08:27:39.674881+0000","last_became_peered":"2024-03-08T08:27:39.674881+0000","last_unstale":"2024-03-08T08:27:42.285354+0000","last_undegraded":"2024-03-08T08:27:42.285354+0000","last_fullsized":"2024-03-08T08:27:42.285354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664467+0000","last_change":"2024-03-08T08:27:38.656182+0000","last_active":"2024-03-08T08:27:40.664467+0000","last_peered":"2024-03-08T08:27:40.664467+0000","last_clean":"2024-03-08T08:27:40.664467+0000","last_became_active":"2024-03-08T08:27:38.655315+0000","last_became_peered":"2024-03-08T08:27:38.655315+0000","last_unstale":"2024-03-08T08:27:40.664467+0000","last_undegraded":"2024-03-08T08:27:40.664467+0000","last_fullsized":"2024-03-08T08:27:40.664467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664532+0000","last_change":"2024-03-08T08:27:39.666898+0000","last_active":"2024-03-08T08:27:40.664532+0000","last_peered":"2024-03-08T08:27:40.664532+0000","last_clean":"2024-03-08T08:27:40.664532+0000","last_became_active":"2024-03-08T08:27:39.666563+0000","last_became_peered":"2024-03-08T08:27:39.666563+0000","last_unstale":"2024-03-08T08:27:40.664532+0000","last_undegraded":"2024-03-08T08:27:40.664532+0000","last_fullsized":"2024-03-08T08:27:40.664532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229594+0000","last_change":"2024-03-08T08:27:38.656580+0000","last_active":"2024-03-08T08:27:43.229594+0000","last_peered":"2024-03-08T08:27:43.229594+0000","last_clean":"2024-03-08T08:27:43.229594+0000","last_became_active":"2024-03-08T08:27:38.656233+0000","last_became_peered":"2024-03-08T08:27:38.656233+0000","last_unstale":"2024-03-08T08:27:43.229594+0000","last_undegraded":"2024-03-08T08:27:43.229594+0000","last_fullsized":"2024-03-08T08:27:43.229594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285501+0000","last_change":"2024-03-08T08:27:39.677499+0000","last_active":"2024-03-08T08:27:42.285501+0000","last_peered":"2024-03-08T08:27:42.285501+0000","last_clean":"2024-03-08T08:27:42.285501+0000","last_became_active":"2024-03-08T08:27:39.677213+0000","last_became_peered":"2024-03-08T08:27:39.677213+0000","last_unstale":"2024-03-08T08:27:42.285501+0000","last_undegraded":"2024-03-08T08:27:42.285501+0000","last_fullsized":"2024-03-08T08:27:42.285501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:00.627737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665133+0000","last_change":"2024-03-08T08:27:38.653100+0000","last_active":"2024-03-08T08:27:40.665133+0000","last_peered":"2024-03-08T08:27:40.665133+0000","last_clean":"2024-03-08T08:27:40.665133+0000","last_became_active":"2024-03-08T08:27:38.652837+0000","last_became_peered":"2024-03-08T08:27:38.652837+0000","last_unstale":"2024-03-08T08:27:40.665133+0000","last_undegraded":"2024-03-08T08:27:40.665133+0000","last_fullsized":"2024-03-08T08:27:40.665133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285597+0000","last_change":"2024-03-08T08:27:39.673698+0000","last_active":"2024-03-08T08:27:42.285597+0000","last_peered":"2024-03-08T08:27:42.285597+0000","last_clean":"2024-03-08T08:27:42.285597+0000","last_became_active":"2024-03-08T08:27:39.673424+0000","last_became_peered":"2024-03-08T08:27:39.673424+0000","last_unstale":"2024-03-08T08:27:42.285597+0000","last_undegraded":"2024-03-08T08:27:42.285597+0000","last_fullsized":"2024-03-08T08:27:42.285597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:12:43.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228967+0000","last_change":"2024-03-08T08:27:38.660277+0000","last_active":"2024-03-08T08:27:43.228967+0000","last_peered":"2024-03-08T08:27:43.228967+0000","last_clean":"2024-03-08T08:27:43.228967+0000","last_became_active":"2024-03-08T08:27:38.659873+0000","last_became_peered":"2024-03-08T08:27:38.659873+0000","last_unstale":"2024-03-08T08:27:43.228967+0000","last_undegraded":"2024-03-08T08:27:43.228967+0000","last_fullsized":"2024-03-08T08:27:43.228967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285694+0000","last_change":"2024-03-08T08:27:39.674403+0000","last_active":"2024-03-08T08:27:42.285694+0000","last_peered":"2024-03-08T08:27:42.285694+0000","last_clean":"2024-03-08T08:27:42.285694+0000","last_became_active":"2024-03-08T08:27:39.674152+0000","last_became_peered":"2024-03-08T08:27:39.674152+0000","last_unstale":"2024-03-08T08:27:42.285694+0000","last_undegraded":"2024-03-08T08:27:42.285694+0000","last_fullsized":"2024-03-08T08:27:42.285694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:58:36.475015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665435+0000","last_change":"2024-03-08T08:27:38.655005+0000","last_active":"2024-03-08T08:27:40.665435+0000","last_peered":"2024-03-08T08:27:40.665435+0000","last_clean":"2024-03-08T08:27:40.665435+0000","last_became_active":"2024-03-08T08:27:38.654683+0000","last_became_peered":"2024-03-08T08:27:38.654683+0000","last_unstale":"2024-03-08T08:27:40.665435+0000","last_undegraded":"2024-03-08T08:27:40.665435+0000","last_fullsized":"2024-03-08T08:27:40.665435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229535+0000","last_change":"2024-03-08T08:27:39.673959+0000","last_active":"2024-03-08T08:27:43.229535+0000","last_peered":"2024-03-08T08:27:43.229535+0000","last_clean":"2024-03-08T08:27:43.229535+0000","last_became_active":"2024-03-08T08:27:39.673369+0000","last_became_peered":"2024-03-08T08:27:39.673369+0000","last_unstale":"2024-03-08T08:27:43.229535+0000","last_undegraded":"2024-03-08T08:27:43.229535+0000","last_fullsized":"2024-03-08T08:27:43.229535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:00.627737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229488+0000","last_change":"2024-03-08T08:27:38.661161+0000","last_active":"2024-03-08T08:27:43.229488+0000","last_peered":"2024-03-08T08:27:43.229488+0000","last_clean":"2024-03-08T08:27:43.229488+0000","last_became_active":"2024-03-08T08:27:38.660876+0000","last_became_peered":"2024-03-08T08:27:38.660876+0000","last_unstale":"2024-03-08T08:27:43.229488+0000","last_undegraded":"2024-03-08T08:27:43.229488+0000","last_fullsized":"2024-03-08T08:27:43.229488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285472+0000","last_change":"2024-03-08T08:27:39.682454+0000","last_active":"2024-03-08T08:27:42.285472+0000","last_peered":"2024-03-08T08:27:42.285472+0000","last_clean":"2024-03-08T08:27:42.285472+0000","last_became_active":"2024-03-08T08:27:39.682178+0000","last_became_peered":"2024-03-08T08:27:39.682178+0000","last_unstale":"2024-03-08T08:27:42.285472+0000","last_undegraded":"2024-03-08T08:27:42.285472+0000","last_fullsized":"2024-03-08T08:27:42.285472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:07.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285074+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:27:42.285074+0000","last_peered":"2024-03-08T08:27:42.285074+0000","last_clean":"2024-03-08T08:27:42.285074+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:27:42.285074+0000","last_undegraded":"2024-03-08T08:27:42.285074+0000","last_fullsized":"2024-03-08T08:27:42.285074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229052+0000","last_change":"2024-03-08T08:27:38.659294+0000","last_active":"2024-03-08T08:27:43.229052+0000","last_peered":"2024-03-08T08:27:43.229052+0000","last_clean":"2024-03-08T08:27:43.229052+0000","last_became_active":"2024-03-08T08:27:38.659037+0000","last_became_peered":"2024-03-08T08:27:38.659037+0000","last_unstale":"2024-03-08T08:27:43.229052+0000","last_undegraded":"2024-03-08T08:27:43.229052+0000","last_fullsized":"2024-03-08T08:27:43.229052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228947+0000","last_change":"2024-03-08T08:27:39.681265+0000","last_active":"2024-03-08T08:27:43.228947+0000","last_peered":"2024-03-08T08:27:43.228947+0000","last_clean":"2024-03-08T08:27:43.228947+0000","last_became_active":"2024-03-08T08:27:39.680995+0000","last_became_peered":"2024-03-08T08:27:39.680995+0000","last_unstale":"2024-03-08T08:27:43.228947+0000","last_undegraded":"2024-03-08T08:27:43.228947+0000","last_fullsized":"2024-03-08T08:27:43.228947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.284946+0000","last_change":"2024-03-08T08:27:38.663815+0000","last_active":"2024-03-08T08:27:42.284946+0000","last_peered":"2024-03-08T08:27:42.284946+0000","last_clean":"2024-03-08T08:27:42.284946+0000","last_became_active":"2024-03-08T08:27:38.663589+0000","last_became_peered":"2024-03-08T08:27:38.663589+0000","last_unstale":"2024-03-08T08:27:42.284946+0000","last_undegraded":"2024-03-08T08:27:42.284946+0000","last_fullsized":"2024-03-08T08:27:42.284946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229606+0000","last_change":"2024-03-08T08:27:39.673596+0000","last_active":"2024-03-08T08:27:43.229606+0000","last_peered":"2024-03-08T08:27:43.229606+0000","last_clean":"2024-03-08T08:27:43.229606+0000","last_became_active":"2024-03-08T08:27:39.673171+0000","last_became_peered":"2024-03-08T08:27:39.673171+0000","last_unstale":"2024-03-08T08:27:43.229606+0000","last_undegraded":"2024-03-08T08:27:43.229606+0000","last_fullsized":"2024-03-08T08:27:43.229606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229195+0000","last_change":"2024-03-08T08:27:38.658281+0000","last_active":"2024-03-08T08:27:43.229195+0000","last_peered":"2024-03-08T08:27:43.229195+0000","last_clean":"2024-03-08T08:27:43.229195+0000","last_became_active":"2024-03-08T08:27:38.657983+0000","last_became_peered":"2024-03-08T08:27:38.657983+0000","last_unstale":"2024-03-08T08:27:43.229195+0000","last_undegraded":"2024-03-08T08:27:43.229195+0000","last_fullsized":"2024-03-08T08:27:43.229195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.662315+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.661362+0000","last_became_peered":"2024-03-08T08:27:39.661362+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229349+0000","last_change":"2024-03-08T08:27:38.659980+0000","last_active":"2024-03-08T08:27:43.229349+0000","last_peered":"2024-03-08T08:27:43.229349+0000","last_clean":"2024-03-08T08:27:43.229349+0000","last_became_active":"2024-03-08T08:27:38.659704+0000","last_became_peered":"2024-03-08T08:27:38.659704+0000","last_unstale":"2024-03-08T08:27:43.229349+0000","last_undegraded":"2024-03-08T08:27:43.229349+0000","last_fullsized":"2024-03-08T08:27:43.229349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665238+0000","last_change":"2024-03-08T08:27:39.678619+0000","last_active":"2024-03-08T08:27:40.665238+0000","last_peered":"2024-03-08T08:27:40.665238+0000","last_clean":"2024-03-08T08:27:40.665238+0000","last_became_active":"2024-03-08T08:27:39.678371+0000","last_became_peered":"2024-03-08T08:27:39.678371+0000","last_unstale":"2024-03-08T08:27:40.665238+0000","last_undegraded":"2024-03-08T08:27:40.665238+0000","last_fullsized":"2024-03-08T08:27:40.665238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665304+0000","last_change":"2024-03-08T08:27:38.654132+0000","last_active":"2024-03-08T08:27:40.665304+0000","last_peered":"2024-03-08T08:27:40.665304+0000","last_clean":"2024-03-08T08:27:40.665304+0000","last_became_active":"2024-03-08T08:27:38.653844+0000","last_became_peered":"2024-03-08T08:27:38.653844+0000","last_unstale":"2024-03-08T08:27:40.665304+0000","last_undegraded":"2024-03-08T08:27:40.665304+0000","last_fullsized":"2024-03-08T08:27:40.665304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229651+0000","last_change":"2024-03-08T08:27:39.679299+0000","last_active":"2024-03-08T08:27:43.229651+0000","last_peered":"2024-03-08T08:27:43.229651+0000","last_clean":"2024-03-08T08:27:43.229651+0000","last_became_active":"2024-03-08T08:27:39.679023+0000","last_became_peered":"2024-03-08T08:27:39.679023+0000","last_unstale":"2024-03-08T08:27:43.229651+0000","last_undegraded":"2024-03-08T08:27:43.229651+0000","last_fullsized":"2024-03-08T08:27:43.229651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:12:43.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285620+0000","last_change":"2024-03-08T08:27:38.665656+0000","last_active":"2024-03-08T08:27:42.285620+0000","last_peered":"2024-03-08T08:27:42.285620+0000","last_clean":"2024-03-08T08:27:42.285620+0000","last_became_active":"2024-03-08T08:27:38.665370+0000","last_became_peered":"2024-03-08T08:27:38.665370+0000","last_unstale":"2024-03-08T08:27:42.285620+0000","last_undegraded":"2024-03-08T08:27:42.285620+0000","last_fullsized":"2024-03-08T08:27:42.285620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665354+0000","last_change":"2024-03-08T08:27:39.664052+0000","last_active":"2024-03-08T08:27:40.665354+0000","last_peered":"2024-03-08T08:27:40.665354+0000","last_clean":"2024-03-08T08:27:40.665354+0000","last_became_active":"2024-03-08T08:27:39.663794+0000","last_became_peered":"2024-03-08T08:27:39.663794+0000","last_unstale":"2024-03-08T08:27:40.665354+0000","last_undegraded":"2024-03-08T08:27:40.665354+0000","last_fullsized":"2024-03-08T08:27:40.665354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:41.113731+0000","last_change":"2024-03-08T08:27:38.658214+0000","last_active":"2024-03-08T08:27:41.113731+0000","last_peered":"2024-03-08T08:27:41.113731+0000","last_clean":"2024-03-08T08:27:41.113731+0000","last_became_active":"2024-03-08T08:27:38.657944+0000","last_became_peered":"2024-03-08T08:27:38.657944+0000","last_unstale":"2024-03-08T08:27:41.113731+0000","last_undegraded":"2024-03-08T08:27:41.113731+0000","last_fullsized":"2024-03-08T08:27:41.113731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229036+0000","last_change":"2024-03-08T08:27:39.671410+0000","last_active":"2024-03-08T08:27:43.229036+0000","last_peered":"2024-03-08T08:27:43.229036+0000","last_clean":"2024-03-08T08:27:43.229036+0000","last_became_active":"2024-03-08T08:27:39.671169+0000","last_became_peered":"2024-03-08T08:27:39.671169+0000","last_unstale":"2024-03-08T08:27:43.229036+0000","last_undegraded":"2024-03-08T08:27:43.229036+0000","last_fullsized":"2024-03-08T08:27:43.229036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665140+0000","last_change":"2024-03-08T08:27:38.657833+0000","last_active":"2024-03-08T08:27:40.665140+0000","last_peered":"2024-03-08T08:27:40.665140+0000","last_clean":"2024-03-08T08:27:40.665140+0000","last_became_active":"2024-03-08T08:27:38.657560+0000","last_became_peered":"2024-03-08T08:27:38.657560+0000","last_unstale":"2024-03-08T08:27:40.665140+0000","last_undegraded":"2024-03-08T08:27:40.665140+0000","last_fullsized":"2024-03-08T08:27:40.665140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230104+0000","last_change":"2024-03-08T08:27:39.668821+0000","last_active":"2024-03-08T08:27:43.230104+0000","last_peered":"2024-03-08T08:27:43.230104+0000","last_clean":"2024-03-08T08:27:43.230104+0000","last_became_active":"2024-03-08T08:27:39.668559+0000","last_became_peered":"2024-03-08T08:27:39.668559+0000","last_unstale":"2024-03-08T08:27:43.230104+0000","last_undegraded":"2024-03-08T08:27:43.230104+0000","last_fullsized":"2024-03-08T08:27:43.230104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.284921+0000","last_change":"2024-03-08T08:27:38.665148+0000","last_active":"2024-03-08T08:27:42.284921+0000","last_peered":"2024-03-08T08:27:42.284921+0000","last_clean":"2024-03-08T08:27:42.284921+0000","last_became_active":"2024-03-08T08:27:38.664834+0000","last_became_peered":"2024-03-08T08:27:38.664834+0000","last_unstale":"2024-03-08T08:27:42.284921+0000","last_undegraded":"2024-03-08T08:27:42.284921+0000","last_fullsized":"2024-03-08T08:27:42.284921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.667641+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.667296+0000","last_became_peered":"2024-03-08T08:27:39.667296+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665163+0000","last_change":"2024-03-08T08:27:38.660688+0000","last_active":"2024-03-08T08:27:40.665163+0000","last_peered":"2024-03-08T08:27:40.665163+0000","last_clean":"2024-03-08T08:27:40.665163+0000","last_became_active":"2024-03-08T08:27:38.660427+0000","last_became_peered":"2024-03-08T08:27:38.660427+0000","last_unstale":"2024-03-08T08:27:40.665163+0000","last_undegraded":"2024-03-08T08:27:40.665163+0000","last_fullsized":"2024-03-08T08:27:40.665163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228902+0000","last_change":"2024-03-08T08:27:39.665202+0000","last_active":"2024-03-08T08:27:43.228902+0000","last_peered":"2024-03-08T08:27:43.228902+0000","last_clean":"2024-03-08T08:27:43.228902+0000","last_became_active":"2024-03-08T08:27:39.663567+0000","last_became_peered":"2024-03-08T08:27:39.663567+0000","last_unstale":"2024-03-08T08:27:43.228902+0000","last_undegraded":"2024-03-08T08:27:43.228902+0000","last_fullsized":"2024-03-08T08:27:43.228902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665293+0000","last_change":"2024-03-08T08:27:38.660223+0000","last_active":"2024-03-08T08:27:40.665293+0000","last_peered":"2024-03-08T08:27:40.665293+0000","last_clean":"2024-03-08T08:27:40.665293+0000","last_became_active":"2024-03-08T08:27:38.659939+0000","last_became_peered":"2024-03-08T08:27:38.659939+0000","last_unstale":"2024-03-08T08:27:40.665293+0000","last_undegraded":"2024-03-08T08:27:40.665293+0000","last_fullsized":"2024-03-08T08:27:40.665293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228969+0000","last_change":"2024-03-08T08:27:39.670191+0000","last_active":"2024-03-08T08:27:43.228969+0000","last_peered":"2024-03-08T08:27:43.228969+0000","last_clean":"2024-03-08T08:27:43.228969+0000","last_became_active":"2024-03-08T08:27:39.669921+0000","last_became_peered":"2024-03-08T08:27:39.669921+0000","last_unstale":"2024-03-08T08:27:43.228969+0000","last_undegraded":"2024-03-08T08:27:43.228969+0000","last_fullsized":"2024-03-08T08:27:43.228969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285389+0000","last_change":"2024-03-08T08:27:38.665118+0000","last_active":"2024-03-08T08:27:42.285389+0000","last_peered":"2024-03-08T08:27:42.285389+0000","last_clean":"2024-03-08T08:27:42.285389+0000","last_became_active":"2024-03-08T08:27:38.664812+0000","last_became_peered":"2024-03-08T08:27:38.664812+0000","last_unstale":"2024-03-08T08:27:42.285389+0000","last_undegraded":"2024-03-08T08:27:42.285389+0000","last_fullsized":"2024-03-08T08:27:42.285389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664566+0000","last_change":"2024-03-08T08:27:39.672103+0000","last_active":"2024-03-08T08:27:40.664566+0000","last_peered":"2024-03-08T08:27:40.664566+0000","last_clean":"2024-03-08T08:27:40.664566+0000","last_became_active":"2024-03-08T08:27:39.671821+0000","last_became_peered":"2024-03-08T08:27:39.671821+0000","last_unstale":"2024-03-08T08:27:40.664566+0000","last_undegraded":"2024-03-08T08:27:40.664566+0000","last_fullsized":"2024-03-08T08:27:40.664566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229542+0000","last_change":"2024-03-08T08:27:39.676821+0000","last_active":"2024-03-08T08:27:43.229542+0000","last_peered":"2024-03-08T08:27:43.229542+0000","last_clean":"2024-03-08T08:27:43.229542+0000","last_became_active":"2024-03-08T08:27:39.674571+0000","last_became_peered":"2024-03-08T08:27:39.674571+0000","last_unstale":"2024-03-08T08:27:43.229542+0000","last_undegraded":"2024-03-08T08:27:43.229542+0000","last_fullsized":"2024-03-08T08:27:43.229542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:58.646243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285171+0000","last_change":"2024-03-08T08:27:38.667618+0000","last_active":"2024-03-08T08:27:42.285171+0000","last_peered":"2024-03-08T08:27:42.285171+0000","last_clean":"2024-03-08T08:27:42.285171+0000","last_became_active":"2024-03-08T08:27:38.667361+0000","last_became_peered":"2024-03-08T08:27:38.667361+0000","last_unstale":"2024-03-08T08:27:42.285171+0000","last_undegraded":"2024-03-08T08:27:42.285171+0000","last_fullsized":"2024-03-08T08:27:42.285171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229446+0000","last_change":"2024-03-08T08:27:39.679501+0000","last_active":"2024-03-08T08:27:43.229446+0000","last_peered":"2024-03-08T08:27:43.229446+0000","last_clean":"2024-03-08T08:27:43.229446+0000","last_became_active":"2024-03-08T08:27:39.679204+0000","last_became_peered":"2024-03-08T08:27:39.679204+0000","last_unstale":"2024-03-08T08:27:43.229446+0000","last_undegraded":"2024-03-08T08:27:43.229446+0000","last_fullsized":"2024-03-08T08:27:43.229446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285442+0000","last_change":"2024-03-08T08:27:38.667616+0000","last_active":"2024-03-08T08:27:42.285442+0000","last_peered":"2024-03-08T08:27:42.285442+0000","last_clean":"2024-03-08T08:27:42.285442+0000","last_became_active":"2024-03-08T08:27:38.667335+0000","last_became_peered":"2024-03-08T08:27:38.667335+0000","last_unstale":"2024-03-08T08:27:42.285442+0000","last_undegraded":"2024-03-08T08:27:42.285442+0000","last_fullsized":"2024-03-08T08:27:42.285442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229948+0000","last_change":"2024-03-08T08:27:39.681042+0000","last_active":"2024-03-08T08:27:43.229948+0000","last_peered":"2024-03-08T08:27:43.229948+0000","last_clean":"2024-03-08T08:27:43.229948+0000","last_became_active":"2024-03-08T08:27:39.680791+0000","last_became_peered":"2024-03-08T08:27:39.680791+0000","last_unstale":"2024-03-08T08:27:43.229948+0000","last_undegraded":"2024-03-08T08:27:43.229948+0000","last_fullsized":"2024-03-08T08:27:43.229948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229414+0000","last_change":"2024-03-08T08:27:38.659145+0000","last_active":"2024-03-08T08:27:43.229414+0000","last_peered":"2024-03-08T08:27:43.229414+0000","last_clean":"2024-03-08T08:27:43.229414+0000","last_became_active":"2024-03-08T08:27:38.658848+0000","last_became_peered":"2024-03-08T08:27:38.658848+0000","last_unstale":"2024-03-08T08:27:43.229414+0000","last_undegraded":"2024-03-08T08:27:43.229414+0000","last_fullsized":"2024-03-08T08:27:43.229414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229123+0000","last_change":"2024-03-08T08:27:39.668041+0000","last_active":"2024-03-08T08:27:43.229123+0000","last_peered":"2024-03-08T08:27:43.229123+0000","last_clean":"2024-03-08T08:27:43.229123+0000","last_became_active":"2024-03-08T08:27:39.667724+0000","last_became_peered":"2024-03-08T08:27:39.667724+0000","last_unstale":"2024-03-08T08:27:43.229123+0000","last_undegraded":"2024-03-08T08:27:43.229123+0000","last_fullsized":"2024-03-08T08:27:43.229123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230114+0000","last_change":"2024-03-08T08:27:38.657913+0000","last_active":"2024-03-08T08:27:43.230114+0000","last_peered":"2024-03-08T08:27:43.230114+0000","last_clean":"2024-03-08T08:27:43.230114+0000","last_became_active":"2024-03-08T08:27:38.657593+0000","last_became_peered":"2024-03-08T08:27:38.657593+0000","last_unstale":"2024-03-08T08:27:43.230114+0000","last_undegraded":"2024-03-08T08:27:43.230114+0000","last_fullsized":"2024-03-08T08:27:43.230114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:27:47.312 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-08T08:27:47.312 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-08T08:27:47.313 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-08T08:27:47.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-08T08:27:47.313 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-08T08:27:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.446+0000 7f12045c7640 1 Processor -- start 2024-03-08T08:27:47.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.446+0000 7f12045c7640 1 -- start start 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.447+0000 7f12045c7640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f11fc15bea0 0x7f11fc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.447+0000 7f12045c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11fc0b5c30 0x7f11fc15c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.447+0000 7f12045c7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc15f7b0 con 0x7f11fc15df30 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.447+0000 7f12045c7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc15e710 con 0x7f11fc0b5c30 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.447+0000 7f12045c7640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc15e9f0 con 0x7f11fc15bea0 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202b64640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11fc0b5c30 0x7f11fc15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202b64640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11fc0b5c30 0x7f11fc15c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59226/0 (socket says 172.21.15.105:59226) 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202b64640 1 -- 172.21.15.105:0/3535026668 learned_addr learned my addr 172.21.15.105:0/3535026668 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202363640 1 --2- 172.21.15.105:0/3535026668 >> v2:172.21.15.144:3300/0 conn(0x7f11fc15bea0 0x7f11fc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202b64640 1 -- 172.21.15.105:0/3535026668 >> v2:172.21.15.144:3300/0 conn(0x7f11fc15bea0 msgr2=0x7f11fc0b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:47.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202b64640 1 --2- 172.21.15.105:0/3535026668 >> v2:172.21.15.144:3300/0 conn(0x7f11fc15bea0 0x7f11fc0b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202b64640 1 -- 172.21.15.105:0/3535026668 >> v1:172.21.15.144:6789/0 conn(0x7f11fc15df30 legacy=0x7f11fc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202b64640 1 -- 172.21.15.105:0/3535026668 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc15ecd0 con 0x7f11fc0b5c30 2024-03-08T08:27:47.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.448+0000 7f1202b64640 1 --2- 172.21.15.105:0/3535026668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc15c280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f11fc072010 tx=0x7f11f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef70192b936ca09 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f1201361640 1 -- 172.21.15.105:0/3535026668 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4034070 con 0x7f11fc0b5c30 2024-03-08T08:27:47.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f1201361640 1 -- 172.21.15.105:0/3535026668 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4030860 con 0x7f11fc0b5c30 2024-03-08T08:27:47.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f1201361640 1 -- 172.21.15.105:0/3535026668 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4030c30 con 0x7f11fc0b5c30 2024-03-08T08:27:47.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f12045c7640 1 -- 172.21.15.105:0/3535026668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 msgr2=0x7f11fc15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f12045c7640 1 --2- 172.21.15.105:0/3535026668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc15c280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f11fc072010 tx=0x7f11f402b9a0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f12045c7640 1 -- 172.21.15.105:0/3535026668 shutdown_connections 2024-03-08T08:27:47.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f12045c7640 1 --2- 172.21.15.105:0/3535026668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc15c280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f12045c7640 1 --2- 172.21.15.105:0/3535026668 >> v2:172.21.15.144:3300/0 conn(0x7f11fc15bea0 0x7f11fc0b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f12045c7640 1 -- 172.21.15.105:0/3535026668 >> 172.21.15.105:0/3535026668 conn(0x7f11fc09aa00 msgr2=0x7f11fc09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:47.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f12045c7640 1 -- 172.21.15.105:0/3535026668 shutdown_connections 2024-03-08T08:27:47.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.449+0000 7f12045c7640 1 -- 172.21.15.105:0/3535026668 wait complete. 2024-03-08T08:27:47.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.450+0000 7f12045c7640 1 Processor -- start 2024-03-08T08:27:47.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.450+0000 7f12045c7640 1 -- start start 2024-03-08T08:27:47.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f12045c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc14c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f12045c7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11fc15bea0 0x7f11fc1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f12045c7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11fc15df30 0x7f11fc15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f12045c7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc09cb00 con 0x7f11fc0b5c30 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f12045c7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15f290 con 0x7f11fc15bea0 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f12045c7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15e3b0 con 0x7f11fc15df30 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202363640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc14c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202363640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc14c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59230/0 (socket says 172.21.15.105:59230) 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202363640 1 -- 172.21.15.105:0/2952598653 learned_addr learned my addr 172.21.15.105:0/2952598653 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202b64640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11fc15df30 0x7f11fc15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1201b62640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11fc15bea0 0x7f11fc1f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202363640 1 -- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11fc15df30 msgr2=0x7f11fc15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202363640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11fc15df30 0x7f11fc15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202363640 1 -- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11fc15bea0 msgr2=0x7f11fc1f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202363640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11fc15bea0 0x7f11fc1f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1202363640 1 -- 172.21.15.105:0/2952598653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc1f5b80 con 0x7f11fc0b5c30 2024-03-08T08:27:47.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.451+0000 7f1201b62640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11fc15bea0 0x7f11fc1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.452+0000 7f1202363640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc14c480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11f8007ef0 tx=0x7f11f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.452+0000 7f11f2ffd640 1 -- 172.21.15.105:0/2952598653 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8014620 con 0x7f11fc0b5c30 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.452+0000 7f11f2ffd640 1 -- 172.21.15.105:0/2952598653 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f8019070 con 0x7f11fc0b5c30 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.452+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc1f5e60 con 0x7f11fc0b5c30 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.452+0000 7f11f2ffd640 1 -- 172.21.15.105:0/2952598653 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8014a00 con 0x7f11fc0b5c30 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.452+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc14c9c0 con 0x7f11fc0b5c30 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.453+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc09fb80 con 0x7f11fc0b5c30 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.453+0000 7f11f2ffd640 1 -- 172.21.15.105:0/2952598653 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11f8014ba0 con 0x7f11fc0b5c30 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.453+0000 7f11f2ffd640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f11e006b4d0 0x7f11e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.453+0000 7f11f2ffd640 1 -- 172.21.15.105:0/2952598653 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11f808a9e0 con 0x7f11fc0b5c30 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.453+0000 7f1201b62640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f11e006b4d0 0x7f11e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.454+0000 7f1201b62640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f11e006b4d0 0x7f11e006d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11fc15d930 tx=0x7f11ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.458+0000 7f11f2ffd640 1 -- 172.21.15.105:0/2952598653 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11fc09fb80 con 0x7f11fc0b5c30 2024-03-08T08:27:47.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.677+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f11fc075120 con 0x7f11fc0b5c30 2024-03-08T08:27:47.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.680+0000 7f11f2ffd640 1 -- 172.21.15.105:0/2952598653 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171790 (secure 0 0 0) 0x7f11f80598f0 con 0x7f11fc0b5c30 2024-03-08T08:27:47.685 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f11e006b4d0 msgr2=0x7f11e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f11e006b4d0 0x7f11e006d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11fc15d930 tx=0x7f11ec009040 comp rx=0 tx=0).stop 2024-03-08T08:27:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 msgr2=0x7f11fc14c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc14c480 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11f8007ef0 tx=0x7f11f800e4b0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 shutdown_connections 2024-03-08T08:27:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f11e006b4d0 0x7f11e006d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11fc15df30 0x7f11fc15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11fc15bea0 0x7f11fc1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 --2- 172.21.15.105:0/2952598653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11fc0b5c30 0x7f11fc14c480 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 >> 172.21.15.105:0/2952598653 conn(0x7f11fc09aa00 msgr2=0x7f11fc06ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 shutdown_connections 2024-03-08T08:27:47.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.683+0000 7f12045c7640 1 -- 172.21.15.105:0/2952598653 wait complete. 2024-03-08T08:27:47.696 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":3146407311},{"type":"v1","addr":"172.21.15.105:6833","nonce":3146407311}]},"active_addr":"172.21.15.105:6833/3146407311","active_change":"2024-03-08T08:27:19.087732+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":810700160}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1712570918}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":966459860}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2889226494}]}]} 2024-03-08T08:27:47.699 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-08T08:27:47.699 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-08T08:27:47.699 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:47.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.796+0000 7f7bdcf2e640 1 Processor -- start 2024-03-08T08:27:47.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.796+0000 7f7bdcf2e640 1 -- start start 2024-03-08T08:27:47.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bdcf2e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7bd8161520 0x7f7bd80b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bdcf2e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80ce7c0 0x7f7bd80b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bdcf2e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd80b29e0 con 0x7f7bd80ce330 2024-03-08T08:27:47.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bdcf2e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd80d1930 con 0x7f7bd8161520 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bdcf2e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd806c770 con 0x7f7bd80ce7c0 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bcffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7bd80ce330 0x7f7bd80b14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40268/0 (socket says 172.21.15.105:40268) 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bcffff640 1 -- 172.21.15.105:0/2880890302 learned_addr learned my addr 172.21.15.105:0/2880890302 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157198916 0 0) 0x7f7bd80b29e0 con 0x7f7bd80ce330 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd6d9d640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80ce7c0 0x7f7bd80b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc4003520 con 0x7f7bd80ce330 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd659c640 1 --2- 172.21.15.105:0/2880890302 >> v2:172.21.15.144:3300/0 conn(0x7f7bd8161520 0x7f7bd80b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4148328238 0 0) 0x7f7bc4003520 con 0x7f7bd80ce330 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80ce7c0 msgr2=0x7f7bd80b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80ce7c0 0x7f7bd80b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 >> v2:172.21.15.144:3300/0 conn(0x7f7bd8161520 msgr2=0x7f7bd80b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 --2- 172.21.15.105:0/2880890302 >> v2:172.21.15.144:3300/0 conn(0x7f7bd8161520 0x7f7bd80b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd80cec50 con 0x7f7bd80ce330 2024-03-08T08:27:47.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f7bc0001670 con 0x7f7bd80ce330 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd659c640 1 --2- 172.21.15.105:0/2880890302 >> v2:172.21.15.144:3300/0 conn(0x7f7bd8161520 0x7f7bd80b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 >> v1:172.21.15.105:6789/0 conn(0x7f7bd80ce330 legacy=0x7f7bd80b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc40039b0 0x7f7bc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80ce7c0 0x7f7bc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4004d40 0x7f7bc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80d1930 con 0x7f7bc40039b0 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd806c770 con 0x7f7bc4004d40 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc400b9c0 con 0x7f7bd80ce7c0 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.797+0000 7f7bd659c640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc40039b0 0x7f7bc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd6d9d640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4004d40 0x7f7bc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bcffff640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80ce7c0 0x7f7bc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd659c640 1 -- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80ce7c0 msgr2=0x7f7bc4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd659c640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80ce7c0 0x7f7bc4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd659c640 1 -- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4004d40 msgr2=0x7f7bc40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd659c640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4004d40 0x7f7bc40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd659c640 1 -- 172.21.15.105:0/2880890302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc4003520 con 0x7f7bc40039b0 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd6d9d640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4004d40 0x7f7bc40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd659c640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc40039b0 0x7f7bc4003dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80098d0 tx=0x7f7bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab33e6798bf3886c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800f040 con 0x7f7bc40039b0 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8004230 con 0x7f7bc40039b0 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.798+0000 7f7bd5d9b640 1 -- 172.21.15.105:0/2880890302 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8003610 con 0x7f7bc40039b0 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.799+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/2880890302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc40039b0 msgr2=0x7f7bc4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.799+0000 7f7bdcf2e640 1 --2- 172.21.15.105:0/2880890302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc40039b0 0x7f7bc4003dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80098d0 tx=0x7f7bc4005830 comp rx=0 tx=0).stop 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.799+0000 7f7bd6d9d640 1 -- 172.21.15.105:0/2880890302 reap_dead start 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.799+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/2880890302 shutdown_connections 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.799+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/2880890302 >> 172.21.15.105:0/2880890302 conn(0x7f7bd80b36a0 msgr2=0x7f7bd80b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.799+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/2880890302 shutdown_connections 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.799+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/2880890302 wait complete. 2024-03-08T08:27:47.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.800+0000 7f7bdcf2e640 1 Processor -- start 2024-03-08T08:27:47.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.800+0000 7f7bdcf2e640 1 -- start start 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bdcf2e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc40039b0 0x7f7bd81f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bdcf2e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80c76f0 0x7f7bd80b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bdcf2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd80c6d20 0x7f7bd80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bdcf2e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8161900 con 0x7f7bd80c6d20 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bdcf2e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd806ca40 con 0x7f7bc40039b0 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bdcf2e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80b32d0 con 0x7f7bd80c76f0 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd80c6d20 0x7f7bd80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd80c6d20 0x7f7bd80b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59258/0 (socket says 172.21.15.105:59258) 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd6d9d640 1 -- 172.21.15.105:0/1635462183 learned_addr learned my addr 172.21.15.105:0/1635462183 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bcffff640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80c76f0 0x7f7bd80b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd659c640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc40039b0 0x7f7bd81f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd6d9d640 1 -- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80c76f0 msgr2=0x7f7bd80b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd6d9d640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80c76f0 0x7f7bd80b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd6d9d640 1 -- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc40039b0 msgr2=0x7f7bd81f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd6d9d640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc40039b0 0x7f7bd81f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd6d9d640 1 -- 172.21.15.105:0/1635462183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd80c7c60 con 0x7f7bd80c6d20 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.801+0000 7f7bd659c640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc40039b0 0x7f7bd81f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.802+0000 7f7bd6d9d640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd80c6d20 0x7f7bd80b6450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8067970 tx=0x7f7bd0010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.802+0000 7f7bceffd640 1 -- 172.21.15.105:0/1635462183 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0010d80 con 0x7f7bd80c6d20 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.802+0000 7f7bceffd640 1 -- 172.21.15.105:0/1635462183 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd0003990 con 0x7f7bd80c6d20 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.802+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd80c5f60 con 0x7f7bd80c6d20 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.802+0000 7f7bceffd640 1 -- 172.21.15.105:0/1635462183 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0003dc0 con 0x7f7bd80c6d20 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.802+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd80c6520 con 0x7f7bd80c6d20 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.803+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd8075120 con 0x7f7bd80c6d20 2024-03-08T08:27:47.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.803+0000 7f7bceffd640 1 -- 172.21.15.105:0/1635462183 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bd0005d10 con 0x7f7bd80c6d20 2024-03-08T08:27:47.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.803+0000 7f7bceffd640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7bb806b500 0x7f7bb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:47.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.803+0000 7f7bceffd640 1 -- 172.21.15.105:0/1635462183 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bd008a8c0 con 0x7f7bd80c6d20 2024-03-08T08:27:47.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.803+0000 7f7bd659c640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7bb806b500 0x7f7bb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:47.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.804+0000 7f7bd659c640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7bb806b500 0x7f7bb806d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8035000 tx=0x7f7bc8039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:47.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.808+0000 7f7bceffd640 1 -- 172.21.15.105:0/1635462183 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bd0025020 con 0x7f7bd80c6d20 2024-03-08T08:27:47.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.982+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bd806dd80 con 0x7f7bd80c6d20 2024-03-08T08:27:47.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.983+0000 7f7bceffd640 1 -- 172.21.15.105:0/1635462183 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f7bd00596b0 con 0x7f7bd80c6d20 2024-03-08T08:27:47.985 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:47.986 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7bb806b500 msgr2=0x7f7bb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7bb806b500 0x7f7bb806d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8035000 tx=0x7f7bc8039040 comp rx=0 tx=0).stop 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd80c6d20 msgr2=0x7f7bd80b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd80c6d20 0x7f7bd80b6450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8067970 tx=0x7f7bd0010590 comp rx=0 tx=0).stop 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 shutdown_connections 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7bb806b500 0x7f7bb806d9c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bd80c6d20 0x7f7bd80b6450 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bd80c76f0 0x7f7bd80b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 --2- 172.21.15.105:0/1635462183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc40039b0 0x7f7bd81f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 >> 172.21.15.105:0/1635462183 conn(0x7f7bd80b36a0 msgr2=0x7f7bd80cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 shutdown_connections 2024-03-08T08:27:47.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:47.985+0000 7f7bdcf2e640 1 -- 172.21.15.105:0/1635462183 wait complete. 2024-03-08T08:27:47.998 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-08T08:27:47.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:48.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.095+0000 7f3e41f77640 1 Processor -- start 2024-03-08T08:27:48.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.095+0000 7f3e41f77640 1 -- start start 2024-03-08T08:27:48.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e41f77640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3e3c1469e0 0x7f3e3c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e41f77640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e3c06ac80 0x7f3e3c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e41f77640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e3c0adcd0 con 0x7f3e3c0ad870 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e41f77640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c06b610 con 0x7f3e3c06ac80 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e41f77640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c06deb0 con 0x7f3e3c1469e0 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e3c06ac80 0x7f3e3c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e3c06ac80 0x7f3e3c06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59270/0 (socket says 172.21.15.105:59270) 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3bfff640 1 -- 172.21.15.105:0/3544388687 learned_addr learned my addr 172.21.15.105:0/3544388687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/3544388687 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c1469e0 0x7f3e3c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3bfff640 1 -- 172.21.15.105:0/3544388687 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c1469e0 msgr2=0x7f3e3c06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3bfff640 1 --2- 172.21.15.105:0/3544388687 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c1469e0 0x7f3e3c06fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3bfff640 1 -- 172.21.15.105:0/3544388687 >> v1:172.21.15.144:6789/0 conn(0x7f3e3c0ad870 legacy=0x7f3e3c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3bfff640 1 -- 172.21.15.105:0/3544388687 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c06e190 con 0x7f3e3c06ac80 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.096+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/3544388687 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c1469e0 0x7f3e3c06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:48.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.097+0000 7f3e3bfff640 1 --2- 172.21.15.105:0/3544388687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c06b0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c147c30 tx=0x7f3e3402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b321a3977e72953a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.097+0000 7f3e3a7fc640 1 -- 172.21.15.105:0/3544388687 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3402c9c0 con 0x7f3e3c06ac80 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.097+0000 7f3e3a7fc640 1 -- 172.21.15.105:0/3544388687 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3402cb60 con 0x7f3e3c06ac80 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.097+0000 7f3e3a7fc640 1 -- 172.21.15.105:0/3544388687 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e340343f0 con 0x7f3e3c06ac80 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.098+0000 7f3e41f77640 1 -- 172.21.15.105:0/3544388687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 msgr2=0x7f3e3c06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.098+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3544388687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c06b0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c147c30 tx=0x7f3e3402be20 comp rx=0 tx=0).stop 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.098+0000 7f3e41f77640 1 -- 172.21.15.105:0/3544388687 shutdown_connections 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.098+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3544388687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c06b0d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.098+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3544388687 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c1469e0 0x7f3e3c06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.098+0000 7f3e41f77640 1 -- 172.21.15.105:0/3544388687 >> 172.21.15.105:0/3544388687 conn(0x7f3e3c0b2670 msgr2=0x7f3e3c0b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.098+0000 7f3e41f77640 1 -- 172.21.15.105:0/3544388687 shutdown_connections 2024-03-08T08:27:48.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.098+0000 7f3e41f77640 1 -- 172.21.15.105:0/3544388687 wait complete. 2024-03-08T08:27:48.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.099+0000 7f3e41f77640 1 Processor -- start 2024-03-08T08:27:48.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.099+0000 7f3e41f77640 1 -- start start 2024-03-08T08:27:48.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e41f77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e41f77640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c0ad870 0x7f3e3c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e41f77640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1469e0 0x7f3e3c145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e41f77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c06e470 con 0x7f3e3c06ac80 2024-03-08T08:27:48.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e41f77640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c0b45d0 con 0x7f3e3c0ad870 2024-03-08T08:27:48.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e41f77640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c146dc0 con 0x7f3e3c1469e0 2024-03-08T08:27:48.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c1f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59278/0 (socket says 172.21.15.105:59278) 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 -- 172.21.15.105:0/3920871019 learned_addr learned my addr 172.21.15.105:0/3920871019 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3affd640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c0ad870 0x7f3e3c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3bfff640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1469e0 0x7f3e3c145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 -- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1469e0 msgr2=0x7f3e3c145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1469e0 0x7f3e3c145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 -- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c0ad870 msgr2=0x7f3e3c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c0ad870 0x7f3e3c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 -- 172.21.15.105:0/3920871019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c0adea0 con 0x7f3e3c06ac80 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3bfff640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1469e0 0x7f3e3c145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c1f5050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c7f0 tx=0x7f3e2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e23fff640 1 -- 172.21.15.105:0/3920871019 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c015070 con 0x7f3e3c06ac80 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e23fff640 1 -- 172.21.15.105:0/3920871019 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c007a30 con 0x7f3e3c06ac80 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e23fff640 1 -- 172.21.15.105:0/3920871019 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c007e00 con 0x7f3e3c06ac80 2024-03-08T08:27:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c0cde30 con 0x7f3e3c06ac80 2024-03-08T08:27:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.100+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c0ce3f0 con 0x7f3e3c06ac80 2024-03-08T08:27:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.101+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c1443e0 con 0x7f3e3c06ac80 2024-03-08T08:27:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.102+0000 7f3e23fff640 1 -- 172.21.15.105:0/3920871019 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e2c018070 con 0x7f3e3c06ac80 2024-03-08T08:27:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.102+0000 7f3e23fff640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e2806b4d0 0x7f3e2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.102+0000 7f3e23fff640 1 -- 172.21.15.105:0/3920871019 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e2c08a0c0 con 0x7f3e3c06ac80 2024-03-08T08:27:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.102+0000 7f3e3affd640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e2806b4d0 0x7f3e2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.103+0000 7f3e3affd640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e2806b4d0 0x7f3e2806d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c1f73e0 tx=0x7f3e30007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.107+0000 7f3e23fff640 1 -- 172.21.15.105:0/3920871019 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e3c1443e0 con 0x7f3e3c06ac80 2024-03-08T08:27:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.279+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e3c075120 con 0x7f3e3c06ac80 2024-03-08T08:27:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.280+0000 7f3e23fff640 1 -- 172.21.15.105:0/3920871019 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f3e2c0588f0 con 0x7f3e3c06ac80 2024-03-08T08:27:48.284 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:48.284 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e2806b4d0 msgr2=0x7f3e2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e2806b4d0 0x7f3e2806d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c1f73e0 tx=0x7f3e30007e30 comp rx=0 tx=0).stop 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 msgr2=0x7f3e3c1f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c1f5050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c7f0 tx=0x7f3e2c00cbf0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 shutdown_connections 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e2806b4d0 0x7f3e2806d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1469e0 0x7f3e3c145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e3c0ad870 0x7f3e3c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 --2- 172.21.15.105:0/3920871019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c06ac80 0x7f3e3c1f5050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 >> 172.21.15.105:0/3920871019 conn(0x7f3e3c0b2670 msgr2=0x7f3e3c143cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 shutdown_connections 2024-03-08T08:27:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.282+0000 7f3e41f77640 1 -- 172.21.15.105:0/3920871019 wait complete. 2024-03-08T08:27:48.296 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-08T08:27:48.296 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-08T08:27:48.297 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-08T08:27:48.297 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-08T08:27:48.298 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-08T08:27:48.298 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-08T08:27:48.298 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-08T08:27:48.299 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-08T08:27:48.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.465+0000 7fae238ec640 1 Processor -- start 2024-03-08T08:27:48.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.466+0000 7fae238ec640 1 -- start start 2024-03-08T08:27:48.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.466+0000 7f6565ed4640 1 Processor -- start 2024-03-08T08:27:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.466+0000 7f6565ed4640 1 -- start start 2024-03-08T08:27:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae238ec640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae1c166970 0x7fae1c1640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae238ec640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae238ec640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c162e00 con 0x7fae1c0c3210 2024-03-08T08:27:48.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae238ec640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c1630e0 con 0x7fae1c166970 2024-03-08T08:27:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae238ec640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1633c0 con 0x7fae1c1645f0 2024-03-08T08:27:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7ffbca1cc640 1 Processor -- start 2024-03-08T08:27:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7f16c833e640 1 Processor -- start 2024-03-08T08:27:48.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fc02a537640 1 Processor -- start 2024-03-08T08:27:48.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7f5ab6847640 1 Processor -- start 2024-03-08T08:27:48.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7f5ab6847640 1 -- start start 2024-03-08T08:27:48.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae20e87640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae1c166970 0x7fae1c1640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae20e87640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae1c166970 0x7fae1c1640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59576/0 (socket says 172.21.15.105:59576) 2024-03-08T08:27:48.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae20e87640 1 -- 172.21.15.105:0/2843997616 learned_addr learned my addr 172.21.15.105:0/2843997616 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7f16c833e640 1 -- start start 2024-03-08T08:27:48.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7fae21e89640 1 --2- 172.21.15.105:0/2843997616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fae20e87640 1 -- 172.21.15.105:0/2843997616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 msgr2=0x7fae1c164a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fae20e87640 1 --2- 172.21.15.105:0/2843997616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c164a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fae20e87640 1 -- 172.21.15.105:0/2843997616 >> v1:172.21.15.105:6789/0 conn(0x7fae1c0c3210 legacy=0x7fae1c0c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fae20e87640 1 -- 172.21.15.105:0/2843997616 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c1636a0 con 0x7fae1c166970 2024-03-08T08:27:48.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fb58a0df640 1 Processor -- start 2024-03-08T08:27:48.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f5ab6847640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5aa80e01f0 0x7f5aa80398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f5ab6847640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 0x7f5aa80534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f5ab6847640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa8039fd0 con 0x7f5aa803a780 2024-03-08T08:27:48.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f5ab6847640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa8053060 con 0x7f5aa80e01f0 2024-03-08T08:27:48.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f5ab6847640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8052a10 con 0x7f5aa8050e80 2024-03-08T08:27:48.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c833e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b80ead00 0x7f16b804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c833e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f16b804ffd0 0x7f16b8037bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c833e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b8053720 con 0x7f16b8038110 2024-03-08T08:27:48.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c833e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b8054330 con 0x7f16b804ffd0 2024-03-08T08:27:48.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c833e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8053a00 con 0x7f16b80ead00 2024-03-08T08:27:48.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fd07728f640 1 Processor -- start 2024-03-08T08:27:48.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.467+0000 7ffbca1cc640 1 -- start start 2024-03-08T08:27:48.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7ffbca1cc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffbc406d9b0 0x7ffbc4070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fc02a537640 1 -- start start 2024-03-08T08:27:48.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc02a537640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc0241657c0 0x7fc0240d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc02a537640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc024165ba0 0x7fc0240b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc02a537640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0240d4670 con 0x7fc0240d04c0 2024-03-08T08:27:48.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc02a537640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0240d4420 con 0x7fc0241657c0 2024-03-08T08:27:48.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc02a537640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0240cee60 con 0x7fc024165ba0 2024-03-08T08:27:48.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7ffbca1cc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc40719a0 con 0x7ffbc4159240 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f6565ed4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6558009190 0x7f6558052f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f6565ed4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6558048c40 0x7f6558049060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f6565ed4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65580eb2a0 0x7f65580eb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f6565ed4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65580495a0 con 0x7f65580eb2a0 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f6565ed4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6558009570 con 0x7f6558048c40 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f6565ed4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6558035970 con 0x7f6558009190 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f655ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65580eb2a0 0x7f65580eb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f655ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65580eb2a0 0x7f65580eb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59292/0 (socket says 172.21.15.105:59292) 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7f655ffff640 1 -- 172.21.15.105:0/2494403848 learned_addr learned my addr 172.21.15.105:0/2494403848 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f655ffff640 1 -- 172.21.15.105:0/2494403848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6558009190 msgr2=0x7f6558052f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f655ffff640 1 --2- 172.21.15.105:0/2494403848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6558009190 0x7f6558052f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f655ffff640 1 -- 172.21.15.105:0/2494403848 >> v2:172.21.15.144:3300/0 conn(0x7f6558048c40 msgr2=0x7f6558049060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-08T08:27:48.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f655ffff640 1 --2- 172.21.15.105:0/2494403848 >> v2:172.21.15.144:3300/0 conn(0x7f6558048c40 0x7f6558049060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f655ffff640 1 -- 172.21.15.105:0/2494403848 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6558035c30 con 0x7f65580eb2a0 2024-03-08T08:27:48.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7ffbca1cc640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc40b2940 con 0x7ffbc41636c0 2024-03-08T08:27:48.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7ffbca1cc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc4159620 con 0x7ffbc406d9b0 2024-03-08T08:27:48.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7ffbc99cb640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ffbc41636c0 0x7ffbc40ae4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:40834/0 (socket says 172.21.15.105:40834) 2024-03-08T08:27:48.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7ffbc99cb640 1 -- 172.21.15.105:0/1905123342 learned_addr learned my addr 172.21.15.105:0/1905123342 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:48.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f5aaffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5aa80e01f0 0x7f5aa80398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f5aaffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5aa80e01f0 0x7f5aa80398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59320/0 (socket says 172.21.15.105:59320) 2024-03-08T08:27:48.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c60da640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b80ead00 0x7f16b804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c60da640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b80ead00 0x7f16b804fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:48778/0 (socket says 172.21.15.105:48778) 2024-03-08T08:27:48.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c60da640 1 -- 172.21.15.105:0/1878735722 learned_addr learned my addr 172.21.15.105:0/1878735722 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7ffbc89c9640 1 --2- 172.21.15.105:0/1905123342 >> v2:172.21.15.105:3300/0 conn(0x7ffbc406d9b0 0x7ffbc4070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fb58a0df640 1 -- start start 2024-03-08T08:27:48.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fb58a0df640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb57c04fc40 0x7fb57c0517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fb58a0df640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c048550 con 0x7fb57c0081c0 2024-03-08T08:27:48.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fb58a0df640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c0487f0 con 0x7fb57c0548e0 2024-03-08T08:27:48.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fb58a0df640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57c048ab0 con 0x7fb57c04fc40 2024-03-08T08:27:48.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc029535640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc0241657c0 0x7fc0240d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc028d34640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc024165ba0 0x7fc0240b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc029535640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc0241657c0 0x7fc0240d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59322/0 (socket says 172.21.15.105:59322) 2024-03-08T08:27:48.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f5aaffff640 1 -- 172.21.15.105:0/3853474559 learned_addr learned my addr 172.21.15.105:0/3853474559 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc029d36640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc0240d04c0 0x7fc0240cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40292/0 (socket says 172.21.15.105:40292) 2024-03-08T08:27:48.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.468+0000 7fd07728f640 1 -- start start 2024-03-08T08:27:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fc029535640 1 -- 172.21.15.105:0/1950585821 learned_addr learned my addr 172.21.15.105:0/1950585821 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fd07728f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd070161fb0 0x7fd07006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07728f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07006e900 con 0x7fd07006d300 2024-03-08T08:27:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07728f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0700706a0 con 0x7fd0700b36f0 2024-03-08T08:27:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07728f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd070070980 con 0x7fd070161fb0 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559027988 0 0) 0x7ffbc40b2940 con 0x7ffbc41636c0 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb0003520 con 0x7ffbc41636c0 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088526136 0 0) 0x7ffbc40719a0 con 0x7ffbc4159240 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc40b2940 con 0x7ffbc4159240 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3464242409 0 0) 0x7ffbb0003520 con 0x7ffbc41636c0 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 >> v2:172.21.15.105:3300/0 conn(0x7ffbc406d9b0 msgr2=0x7ffbc4070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 --2- 172.21.15.105:0/1905123342 >> v2:172.21.15.105:3300/0 conn(0x7ffbc406d9b0 0x7ffbc4070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 >> v1:172.21.15.105:6789/0 conn(0x7ffbc4159240 legacy=0x7ffbc406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40b2c20 con 0x7ffbc41636c0 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7ffbbc0019f0 con 0x7ffbc41636c0 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 >> v1:172.21.15.144:6789/0 conn(0x7ffbc41636c0 legacy=0x7ffbc40ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb0003c50 0x7ffbb0004050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb0004800 0x7ffbb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb0005210 0x7ffbb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb000c130 con 0x7ffbb0004800 2024-03-08T08:27:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb000c2e0 con 0x7ffbb0005210 2024-03-08T08:27:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb000c4e0 con 0x7ffbb0003c50 2024-03-08T08:27:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fb582ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb57c0081c0 0x7fb57c050020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40294/0 (socket says 172.21.15.105:40294) 2024-03-08T08:27:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fb582ffd640 1 -- 172.21.15.105:0/896017344 learned_addr learned my addr 172.21.15.105:0/896017344 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fb5837fe640 1 --2- 172.21.15.105:0/896017344 >> v2:172.21.15.105:3300/0 conn(0x7fb57c04fc40 0x7fb57c0517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fc029535640 1 -- 172.21.15.105:0/1950585821 >> v2:172.21.15.144:3300/0 conn(0x7fc024165ba0 msgr2=0x7fc0240b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fc029535640 1 --2- 172.21.15.105:0/1950585821 >> v2:172.21.15.144:3300/0 conn(0x7fc024165ba0 0x7fc0240b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fc029535640 1 -- 172.21.15.105:0/1950585821 >> v1:172.21.15.105:6789/0 conn(0x7fc0240d04c0 legacy=0x7fc0240cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fc029535640 1 -- 172.21.15.105:0/1950585821 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0240ce810 con 0x7fc0241657c0 2024-03-08T08:27:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7f5aaffff640 1 -- 172.21.15.105:0/3853474559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 msgr2=0x7f5aa80534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7f16c58d9640 1 --2- 172.21.15.105:0/1878735722 >> v2:172.21.15.105:3300/0 conn(0x7f16b804ffd0 0x7f16b8037bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fae20e87640 1 --2- 172.21.15.105:0/2843997616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae1c166970 0x7fae1c1640b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009870 tx=0x7fae0c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131c0d9ce1139ba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.469+0000 7fae13fff640 1 -- 172.21.15.105:0/2843997616 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c035020 con 0x7fae1c166970 2024-03-08T08:27:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fae13fff640 1 -- 172.21.15.105:0/2843997616 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c031870 con 0x7fae1c166970 2024-03-08T08:27:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fae13fff640 1 -- 172.21.15.105:0/2843997616 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c031c80 con 0x7fae1c166970 2024-03-08T08:27:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07482a640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd07006d300 0x7fd0700ccc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40296/0 (socket says 172.21.15.105:40296) 2024-03-08T08:27:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07482a640 1 -- 172.21.15.105:0/1529734994 learned_addr learned my addr 172.21.15.105:0/1529734994 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07502b640 1 --2- 172.21.15.105:0/1529734994 >> v2:172.21.15.144:3300/0 conn(0x7fd070161fb0 0x7fd07006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd067fff640 1 -- 172.21.15.105:0/1529734994 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591460540 0 0) 0x7fd0700706a0 con 0x7fd0700b36f0 2024-03-08T08:27:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07502b640 1 -- 172.21.15.105:0/1529734994 >> v1:172.21.15.144:6789/0 conn(0x7fd0700b36f0 legacy=0x7fd070071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07502b640 1 -- 172.21.15.105:0/1529734994 >> v1:172.21.15.105:6789/0 conn(0x7fd07006d300 legacy=0x7fd0700ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7fd07502b640 1 -- 172.21.15.105:0/1529734994 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070070c60 con 0x7fd070161fb0 2024-03-08T08:27:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7ffbc91ca640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb0003c50 0x7ffbb0004050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122059853 0 0) 0x7fb57c0487f0 con 0x7fb57c0548e0 2024-03-08T08:27:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fae238ec640 1 -- 172.21.15.105:0/2843997616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae1c166970 msgr2=0x7fae1c1640b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb574003520 con 0x7fb57c0548e0 2024-03-08T08:27:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fae238ec640 1 --2- 172.21.15.105:0/2843997616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae1c166970 0x7fae1c1640b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009870 tx=0x7fae0c02b910 comp rx=0 tx=0).stop 2024-03-08T08:27:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7f5ab55e5640 1 --2- 172.21.15.105:0/3853474559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 0x7f5aa80534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7f5aaeffd640 1 -- 172.21.15.105:0/3853474559 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817784723 0 0) 0x7f5aa8039fd0 con 0x7f5aa803a780 2024-03-08T08:27:48.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7f16c60da640 1 -- 172.21.15.105:0/1878735722 >> v2:172.21.15.105:3300/0 conn(0x7f16b804ffd0 msgr2=0x7f16b8037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7f16c60da640 1 --2- 172.21.15.105:0/1878735722 >> v2:172.21.15.105:3300/0 conn(0x7f16b804ffd0 0x7f16b8037bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7f16c60da640 1 -- 172.21.15.105:0/1878735722 >> v1:172.21.15.144:6789/0 conn(0x7f16b8038110 legacy=0x7f16b80385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.470+0000 7f16c60da640 1 -- 172.21.15.105:0/1878735722 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8053ce0 con 0x7f16b80ead00 2024-03-08T08:27:48.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7f5aaffff640 1 --2- 172.21.15.105:0/3853474559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 0x7f5aa80534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7f5aaffff640 1 -- 172.21.15.105:0/3853474559 >> v1:172.21.15.144:6789/0 conn(0x7f5aa803a780 legacy=0x7f5aa8050770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7f5aaffff640 1 -- 172.21.15.105:0/3853474559 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa8052cf0 con 0x7f5aa80e01f0 2024-03-08T08:27:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7f5ab55e5640 1 --2- 172.21.15.105:0/3853474559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 0x7f5aa80534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fae238ec640 1 -- 172.21.15.105:0/2843997616 shutdown_connections 2024-03-08T08:27:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fae238ec640 1 --2- 172.21.15.105:0/2843997616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c164a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fae238ec640 1 --2- 172.21.15.105:0/2843997616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae1c166970 0x7fae1c1640b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fae238ec640 1 -- 172.21.15.105:0/2843997616 >> 172.21.15.105:0/2843997616 conn(0x7fae1c072110 msgr2=0x7fae1c072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7f16c60da640 1 --2- 172.21.15.105:0/1878735722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b80ead00 0x7f16b804fa90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16c0000b60 tx=0x7f16c00a6d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52e285e5e027bcb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fae238ec640 1 -- 172.21.15.105:0/2843997616 shutdown_connections 2024-03-08T08:27:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 865489505 0 0) 0x7fb574003520 con 0x7fb57c0548e0 2024-03-08T08:27:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 >> v2:172.21.15.105:3300/0 conn(0x7fb57c04fc40 msgr2=0x7fb57c0517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 --2- 172.21.15.105:0/896017344 >> v2:172.21.15.105:3300/0 conn(0x7fb57c04fc40 0x7fb57c0517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7f655ffff640 1 --2- 172.21.15.105:0/2494403848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65580eb2a0 0x7f65580eb6b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6560000b60 tx=0x7f65600b8020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1ac573a6c93921 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 >> v1:172.21.15.105:6789/0 conn(0x7fb57c0081c0 legacy=0x7fb57c050020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7ffbc99cb640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb0005210 0x7ffbb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0530d0 con 0x7fb57c0548e0 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fb570001670 con 0x7fb57c0548e0 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 >> v1:172.21.15.144:6789/0 conn(0x7fb57c0548e0 legacy=0x7fb57c0f11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5740039b0 0x7fb574003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb574004800 0x7fb574004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5740051c0 0x7fb57400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c048ab0 con 0x7fb5740051c0 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57400c010 con 0x7fb5740039b0 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57400c210 con 0x7fb574004800 2024-03-08T08:27:48.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb582ffd640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb574004800 0x7fb574004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7ffbc89c9640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb0004800 0x7ffbb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fb583fff640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5740051c0 0x7fb57400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.471+0000 7fae238ec640 1 -- 172.21.15.105:0/2843997616 wait complete. 2024-03-08T08:27:48.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7f16c50d8640 1 -- 172.21.15.105:0/1878735722 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00b4070 con 0x7f16b80ead00 2024-03-08T08:27:48.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7ffbc89c9640 1 -- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb0003c50 msgr2=0x7ffbb0004050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7ffbc89c9640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbb0003c50 0x7ffbb0004050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7ffbc89c9640 1 -- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb0005210 msgr2=0x7ffbb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7ffbc89c9640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb0005210 0x7ffbb000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7ffbc89c9640 1 -- 172.21.15.105:0/1905123342 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb0003520 con 0x7ffbb0004800 2024-03-08T08:27:48.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fd07502b640 1 --2- 172.21.15.105:0/1529734994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd070161fb0 0x7fd07006e1f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd068009d90 tx=0x7fd06802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f22a1cdc0cf4b621 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7f655e7fc640 1 -- 172.21.15.105:0/2494403848 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65600c5070 con 0x7f65580eb2a0 2024-03-08T08:27:48.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7f655e7fc640 1 -- 172.21.15.105:0/2494403848 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65600b8c80 con 0x7f65580eb2a0 2024-03-08T08:27:48.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7f655e7fc640 1 -- 172.21.15.105:0/2494403848 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65600c5070 con 0x7f65580eb2a0 2024-03-08T08:27:48.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7ffbc99cb640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb0005210 0x7ffbb000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:48.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7ffbc99cb640 1 -- 172.21.15.105:0/1905123342 reap_dead start 2024-03-08T08:27:48.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fc029535640 1 --2- 172.21.15.105:0/1950585821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 0x7fc0240d2500 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc02408c7d0 tx=0x7fc01c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514fb31ac9b1f0cf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fb582ffd640 1 -- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5740039b0 msgr2=0x7fb574003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fb582ffd640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5740039b0 0x7fb574003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fb582ffd640 1 -- 172.21.15.105:0/896017344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5740051c0 msgr2=0x7fb57400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fb582ffd640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5740051c0 0x7fb57400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fb583fff640 1 -- 172.21.15.105:0/896017344 reap_dead start 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fb582ffd640 1 -- 172.21.15.105:0/896017344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0530d0 con 0x7fb574004800 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7f5aaffff640 1 --2- 172.21.15.105:0/3853474559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa80e01f0 0x7f5aa80398f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009870 tx=0x7f5aa402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8486921d2be9aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7f5aaeffd640 1 -- 172.21.15.105:0/3853474559 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa402f830 con 0x7f5aa80e01f0 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f5aaeffd640 1 -- 172.21.15.105:0/3853474559 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa402f9d0 con 0x7f5aa80e01f0 2024-03-08T08:27:48.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f5aaeffd640 1 -- 172.21.15.105:0/3853474559 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa402fda0 con 0x7f5aa80e01f0 2024-03-08T08:27:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fd067fff640 1 -- 172.21.15.105:0/1529734994 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068039070 con 0x7fd070161fb0 2024-03-08T08:27:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.472+0000 7fd067fff640 1 -- 172.21.15.105:0/1529734994 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06802cd10 con 0x7fd070161fb0 2024-03-08T08:27:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 -- 172.21.15.105:0/2494403848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65580eb2a0 msgr2=0x7f65580eb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 --2- 172.21.15.105:0/2494403848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65580eb2a0 0x7f65580eb6b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6560000b60 tx=0x7f65600b8020 comp rx=0 tx=0).stop 2024-03-08T08:27:48.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 -- 172.21.15.105:0/2494403848 shutdown_connections 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 --2- 172.21.15.105:0/2494403848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65580eb2a0 0x7f65580eb6b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 --2- 172.21.15.105:0/2494403848 >> v2:172.21.15.144:3300/0 conn(0x7f6558048c40 0x7f6558049060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 --2- 172.21.15.105:0/2494403848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6558009190 0x7f6558052f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 -- 172.21.15.105:0/2494403848 >> 172.21.15.105:0/2494403848 conn(0x7f655801ef40 msgr2=0x7f6558055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fc01bfff640 1 -- 172.21.15.105:0/1950585821 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c02cb10 con 0x7fc0241657c0 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fc01bfff640 1 -- 172.21.15.105:0/1950585821 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c02ccb0 con 0x7fc0241657c0 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fc01bfff640 1 -- 172.21.15.105:0/1950585821 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c02ce60 con 0x7fc0241657c0 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fae238ec640 1 Processor -- start 2024-03-08T08:27:48.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fae238ec640 1 -- start start 2024-03-08T08:27:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fd07728f640 1 -- 172.21.15.105:0/1529734994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd070161fb0 msgr2=0x7fd07006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fd07728f640 1 --2- 172.21.15.105:0/1529734994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd070161fb0 0x7fd07006e1f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd068009d90 tx=0x7fd06802bf60 comp rx=0 tx=0).stop 2024-03-08T08:27:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f16c50d8640 1 -- 172.21.15.105:0/1878735722 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c00a7b70 con 0x7f16b80ead00 2024-03-08T08:27:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f16c50d8640 1 -- 172.21.15.105:0/1878735722 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00af440 con 0x7f16b80ead00 2024-03-08T08:27:48.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7ffbc89c9640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb0004800 0x7ffbb0004c80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009a90 tx=0x7ffbb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4826605680d7e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fd07728f640 1 -- 172.21.15.105:0/1529734994 shutdown_connections 2024-03-08T08:27:48.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fd07728f640 1 --2- 172.21.15.105:0/1529734994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd070161fb0 0x7fd07006e1f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fd07728f640 1 -- 172.21.15.105:0/1529734994 >> 172.21.15.105:0/1529734994 conn(0x7fd0700b1740 msgr2=0x7fd0700b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 -- 172.21.15.105:0/2494403848 shutdown_connections 2024-03-08T08:27:48.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7f6565ed4640 1 -- 172.21.15.105:0/2494403848 wait complete. 2024-03-08T08:27:48.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7fb582ffd640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb574004800 0x7fb574004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb584000b60 tx=0x7fb574005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52db09ff896ffdf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0039070 con 0x7ffbb0004800 2024-03-08T08:27:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.473+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc002c680 con 0x7ffbb0004800 2024-03-08T08:27:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58404feb0 con 0x7fb574004800 2024-03-08T08:27:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5840a87b0 con 0x7fb574004800 2024-03-08T08:27:48.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fb5827fc640 1 -- 172.21.15.105:0/896017344 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840a8be0 con 0x7fb574004800 2024-03-08T08:27:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae238ec640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae1c0c3210 0x7fae1c14df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f5ab6847640 1 -- 172.21.15.105:0/3853474559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa80e01f0 msgr2=0x7f5aa80398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f5ab6847640 1 --2- 172.21.15.105:0/3853474559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa80e01f0 0x7f5aa80398f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009870 tx=0x7f5aa402ba10 comp rx=0 tx=0).stop 2024-03-08T08:27:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae238ec640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae238ec640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae1c166970 0x7fae1c0c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fc02a537640 1 -- 172.21.15.105:0/1950585821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 msgr2=0x7fc0240d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fc02a537640 1 --2- 172.21.15.105:0/1950585821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 0x7fc0240d2500 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc02408c7d0 tx=0x7fc01c02bf70 comp rx=0 tx=0).stop 2024-03-08T08:27:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae238ec640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1629f0 con 0x7fae1c0c3210 2024-03-08T08:27:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f16c833e640 1 -- 172.21.15.105:0/1878735722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b80ead00 msgr2=0x7f16b804fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae238ec640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c066bb0 con 0x7fae1c166970 2024-03-08T08:27:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae238ec640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c162b70 con 0x7fae1c1645f0 2024-03-08T08:27:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7ffbb7fff640 1 -- 172.21.15.105:0/1905123342 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0009690 con 0x7ffbb0004800 2024-03-08T08:27:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f16c833e640 1 --2- 172.21.15.105:0/1878735722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b80ead00 0x7f16b804fa90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16c0000b60 tx=0x7f16c00a6d00 comp rx=0 tx=0).stop 2024-03-08T08:27:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f16c833e640 1 -- 172.21.15.105:0/1878735722 shutdown_connections 2024-03-08T08:27:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f16c833e640 1 --2- 172.21.15.105:0/1878735722 >> v2:172.21.15.105:3300/0 conn(0x7f16b804ffd0 0x7f16b8037bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f16c833e640 1 --2- 172.21.15.105:0/1878735722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b80ead00 0x7f16b804fa90 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16c0000b60 tx=0x7f16c00a6d00 comp rx=0 tx=0).stop 2024-03-08T08:27:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f16c833e640 1 -- 172.21.15.105:0/1878735722 >> 172.21.15.105:0/1878735722 conn(0x7f16b8009fb0 msgr2=0x7f16b800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f16c833e640 1 -- 172.21.15.105:0/1878735722 shutdown_connections 2024-03-08T08:27:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f16c833e640 1 -- 172.21.15.105:0/1878735722 wait complete. 2024-03-08T08:27:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f5ab6847640 1 -- 172.21.15.105:0/3853474559 shutdown_connections 2024-03-08T08:27:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f5ab6847640 1 --2- 172.21.15.105:0/3853474559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 0x7f5aa80534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f5ab6847640 1 --2- 172.21.15.105:0/3853474559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa80e01f0 0x7f5aa80398f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7f5ab6847640 1 -- 172.21.15.105:0/3853474559 >> 172.21.15.105:0/3853474559 conn(0x7f5aa8054ca0 msgr2=0x7f5aa80550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fd07728f640 1 -- 172.21.15.105:0/1529734994 shutdown_connections 2024-03-08T08:27:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7ffbca1cc640 1 -- 172.21.15.105:0/1905123342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb0004800 msgr2=0x7ffbb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7ffbca1cc640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb0004800 0x7ffbb0004c80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009a90 tx=0x7ffbb0005830 comp rx=0 tx=0).stop 2024-03-08T08:27:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae20e87640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae20e87640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c0c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:48822/0 (socket says 172.21.15.105:48822) 2024-03-08T08:27:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae20e87640 1 -- 172.21.15.105:0/962879581 learned_addr learned my addr 172.21.15.105:0/962879581 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7ffbca1cc640 1 -- 172.21.15.105:0/1905123342 shutdown_connections 2024-03-08T08:27:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7ffbca1cc640 1 --2- 172.21.15.105:0/1905123342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbb0004800 0x7ffbb0004c80 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009a90 tx=0x7ffbb0005830 comp rx=0 tx=0).stop 2024-03-08T08:27:48.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7ffbca1cc640 1 -- 172.21.15.105:0/1905123342 >> 172.21.15.105:0/1905123342 conn(0x7ffbc406ac80 msgr2=0x7ffbc406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fae21e89640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae1c166970 0x7fae1c0c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fb58a0df640 1 -- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb574004800 msgr2=0x7fb574004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fb58a0df640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb574004800 0x7fb574004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb584000b60 tx=0x7fb574005830 comp rx=0 tx=0).stop 2024-03-08T08:27:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7ffbca1cc640 1 -- 172.21.15.105:0/1905123342 shutdown_connections 2024-03-08T08:27:48.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fae20e87640 1 -- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae1c166970 msgr2=0x7fae1c0c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fb58a0df640 1 -- 172.21.15.105:0/896017344 shutdown_connections 2024-03-08T08:27:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fb58a0df640 1 --2- 172.21.15.105:0/896017344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb574004800 0x7fb574004c80 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb584000b60 tx=0x7fb574005830 comp rx=0 tx=0).stop 2024-03-08T08:27:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fb58a0df640 1 -- 172.21.15.105:0/896017344 >> 172.21.15.105:0/896017344 conn(0x7fb57c055380 msgr2=0x7fb57c055790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fae20e87640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae1c166970 0x7fae1c0c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fae20e87640 1 -- 172.21.15.105:0/962879581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae1c0c3210 msgr2=0x7fae1c14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fae20e87640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae1c0c3210 0x7fae1c14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fb58a0df640 1 -- 172.21.15.105:0/896017344 shutdown_connections 2024-03-08T08:27:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7f5ab6847640 1 -- 172.21.15.105:0/3853474559 shutdown_connections 2024-03-08T08:27:48.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fae20e87640 1 -- 172.21.15.105:0/962879581 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c0ada40 con 0x7fae1c1645f0 2024-03-08T08:27:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fd07728f640 1 -- 172.21.15.105:0/1529734994 wait complete. 2024-03-08T08:27:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7f6565ed4640 1 Processor -- start 2024-03-08T08:27:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fc02a537640 1 -- 172.21.15.105:0/1950585821 shutdown_connections 2024-03-08T08:27:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fc02a537640 1 --2- 172.21.15.105:0/1950585821 >> v2:172.21.15.144:3300/0 conn(0x7fc024165ba0 0x7fc0240b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fc02a537640 1 --2- 172.21.15.105:0/1950585821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 0x7fc0240d2500 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.474+0000 7fc02a537640 1 -- 172.21.15.105:0/1950585821 >> 172.21.15.105:0/1950585821 conn(0x7fc02409cdc0 msgr2=0x7fc0240d3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fc02a537640 1 -- 172.21.15.105:0/1950585821 shutdown_connections 2024-03-08T08:27:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fc02a537640 1 -- 172.21.15.105:0/1950585821 wait complete. 2024-03-08T08:27:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7f6565ed4640 1 -- start start 2024-03-08T08:27:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7fb58a0df640 1 -- 172.21.15.105:0/896017344 wait complete. 2024-03-08T08:27:48.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7fd07728f640 1 Processor -- start 2024-03-08T08:27:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7fae20e87640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c0c9bd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0090b0 tx=0x7fae0c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7fd07728f640 1 -- start start 2024-03-08T08:27:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c040070 con 0x7fae1c1645f0 2024-03-08T08:27:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c0add20 con 0x7fae1c1645f0 2024-03-08T08:27:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7f5ab6847640 1 -- 172.21.15.105:0/3853474559 wait complete. 2024-03-08T08:27:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c0ae2e0 con 0x7fae1c1645f0 2024-03-08T08:27:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f5ab6847640 1 Processor -- start 2024-03-08T08:27:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fb58a0df640 1 Processor -- start 2024-03-08T08:27:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f6565ed4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6558009190 0x7f65580dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f6565ed4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6558048c40 0x7f6558007c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f6565ed4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65580eb2a0 0x7f65580e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7f16c833e640 1 Processor -- start 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7f16c833e640 1 -- start start 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c833e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16b8038110 0x7f16b804e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c833e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b804ffd0 0x7f16b8046e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c833e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b8047380 0x7f16b80dbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c833e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80547c0 con 0x7f16b8038110 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c833e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8038cb0 con 0x7f16b804ffd0 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c833e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8054940 con 0x7f16b8047380 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c60da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16b8038110 0x7f16b804e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c60da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16b8038110 0x7f16b804e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59376/0 (socket says 172.21.15.105:59376) 2024-03-08T08:27:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f16c60da640 1 -- 172.21.15.105:0/2211731491 learned_addr learned my addr 172.21.15.105:0/2211731491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fb58a0df640 1 -- start start 2024-03-08T08:27:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7fc02a537640 1 Processor -- start 2024-03-08T08:27:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7fc02a537640 1 -- start start 2024-03-08T08:27:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fc02a537640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240d04c0 0x7fc024070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc02a537640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 0x7fc024153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc02a537640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc024165ba0 0x7fc024158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc02a537640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240d4950 con 0x7fc0241657c0 2024-03-08T08:27:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc02a537640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240d2a40 con 0x7fc0240d04c0 2024-03-08T08:27:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc02a537640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0240c71c0 con 0x7fc024165ba0 2024-03-08T08:27:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f16c58d9640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b804ffd0 0x7f16b8046e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fd07728f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd07006d300 0x7fd0700a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fd07728f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0700b36f0 0x7fd0701643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fd07728f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd070161fb0 0x7fd0700ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fd07728f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd070162390 con 0x7fd07006d300 2024-03-08T08:27:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fd07728f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0700b43d0 con 0x7fd0700b36f0 2024-03-08T08:27:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7fd07728f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd070162510 con 0x7fd070161fb0 2024-03-08T08:27:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07502b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd07006d300 0x7fd0700a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07502b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd07006d300 0x7fd0700a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59372/0 (socket says 172.21.15.105:59372) 2024-03-08T08:27:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07502b640 1 -- 172.21.15.105:0/1080563483 learned_addr learned my addr 172.21.15.105:0/1080563483 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7f5ab6847640 1 -- start start 2024-03-08T08:27:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07582c640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd070161fb0 0x7fd0700ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c030d00 con 0x7fae1c1645f0 2024-03-08T08:27:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c035040 con 0x7fae1c1645f0 2024-03-08T08:27:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029d36640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc024165ba0 0x7fc024158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029d36640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc024165ba0 0x7fc024158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:48832/0 (socket says 172.21.15.105:48832) 2024-03-08T08:27:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029d36640 1 -- 172.21.15.105:0/706690503 learned_addr learned my addr 172.21.15.105:0/706690503 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029535640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240d04c0 0x7fc024070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6558009190 0x7f65580dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f16c60da640 1 -- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b8047380 msgr2=0x7f16b80dbaf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T08:27:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f16c60da640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16b8047380 0x7f16b80dbaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f16c60da640 1 -- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b804ffd0 msgr2=0x7f16b8046e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07482a640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0700b36f0 0x7fd0701643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f16c60da640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16b804ffd0 0x7f16b8046e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f16c60da640 1 -- 172.21.15.105:0/2211731491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b80e8760 con 0x7f16b8038110 2024-03-08T08:27:48.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f655f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6558009190 0x7f65580dbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59386/0 (socket says 172.21.15.105:59386) 2024-03-08T08:27:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f655effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6558048c40 0x7f6558007c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc028d34640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 0x7fc024153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029d36640 1 -- 172.21.15.105:0/706690503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240d04c0 msgr2=0x7fc024070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.475+0000 7ffbca1cc640 1 -- 172.21.15.105:0/1905123342 wait complete. 2024-03-08T08:27:48.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029d36640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240d04c0 0x7fc024070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029d36640 1 -- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 msgr2=0x7fc024153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.476+0000 7ffbca1cc640 1 Processor -- start 2024-03-08T08:27:48.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.477+0000 7ffbca1cc640 1 -- start start 2024-03-08T08:27:48.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f655effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6558048c40 0x7f6558007c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59614/0 (socket says 172.21.15.105:59614) 2024-03-08T08:27:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07582c640 1 -- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0700b36f0 msgr2=0x7fd0701643e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07582c640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0700b36f0 0x7fd0701643e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07582c640 1 -- 172.21.15.105:0/1080563483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd07006d300 msgr2=0x7fd0700a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07582c640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd07006d300 0x7fd0700a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fd07582c640 1 -- 172.21.15.105:0/1080563483 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070200ca0 con 0x7fd070161fb0 2024-03-08T08:27:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029d36640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 0x7fc024153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc029d36640 1 -- 172.21.15.105:0/706690503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02406e360 con 0x7fc024165ba0 2024-03-08T08:27:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7f655f7fe640 1 -- 172.21.15.105:0/2198400472 learned_addr learned my addr 172.21.15.105:0/2198400472 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.478+0000 7fc028d34640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0241657c0 0x7fc024153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f6565ed4640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580364d0 con 0x7f6558009190 2024-03-08T08:27:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f6565ed4640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580361f0 con 0x7f6558048c40 2024-03-08T08:27:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5ab6847640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa803a780 0x7f5aa80492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f655ffff640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65580eb2a0 0x7f65580e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f6565ed4640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580367b0 con 0x7f65580eb2a0 2024-03-08T08:27:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fd07582c640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd070161fb0 0x7fd0700ab1e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00c3e0 tx=0x7fd06c00c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbca1cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbc4156ab0 0x7ffbc415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa803a780 0x7f5aa80492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa803a780 0x7f5aa80492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59402/0 (socket says 172.21.15.105:59402) 2024-03-08T08:27:48.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5ab6847640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 0x7f5aa8180080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fc029d36640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc024165ba0 0x7fc024158e00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c3f0 tx=0x7fc02000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbca1cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc415eff0 0x7ffbc4164b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbca1cc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc41650d0 0x7ffbc4215080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbca1cc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc406d2a0 con 0x7ffbc415eff0 2024-03-08T08:27:48.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbca1cc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40b31e0 con 0x7ffbc4156ab0 2024-03-08T08:27:48.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbca1cc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc406d420 con 0x7ffbc41650d0 2024-03-08T08:27:48.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5aaffff640 1 -- 172.21.15.105:0/3508716458 learned_addr learned my addr 172.21.15.105:0/3508716458 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5ab6847640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa80e27a0 0x7f5aa80e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f16c60da640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16b8038110 0x7f16b804e0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16c00699b0 tx=0x7f16c0068a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fb58a0df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0e63e0 0x7fb57c038360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0200074e0 con 0x7fc024165ba0 2024-03-08T08:27:48.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5ab6847640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8053bb0 con 0x7f5aa803a780 2024-03-08T08:27:48.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5ab6847640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8053a30 con 0x7f5aa80e27a0 2024-03-08T08:27:48.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc020007680 con 0x7fc024165ba0 2024-03-08T08:27:48.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbc89c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc415eff0 0x7ffbc4164b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5ab6847640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8053d30 con 0x7f5aa8050e80 2024-03-08T08:27:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5ab55e5640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa80e27a0 0x7f5aa80e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbc89c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc415eff0 0x7ffbc4164b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59410/0 (socket says 172.21.15.105:59410) 2024-03-08T08:27:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbc89c9640 1 -- 172.21.15.105:0/4265631909 learned_addr learned my addr 172.21.15.105:0/4265631909 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f5aaf7fe640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 0x7f5aa8180080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb58a0df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb57c0dd2e0 0x7fb57c0388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00be070 con 0x7f16b8038110 2024-03-08T08:27:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c00b9670 con 0x7f16b8038110 2024-03-08T08:27:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b80436e0 con 0x7f16b8038110 2024-03-08T08:27:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b8043f20 con 0x7f16b8038110 2024-03-08T08:27:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7ffbc99cb640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc41650d0 0x7ffbc4215080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb58a0df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57c0e67c0 0x7fb57c035f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7ffbc89c9640 1 -- 172.21.15.105:0/4265631909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc41650d0 msgr2=0x7ffbc4215080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb582ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb57c0dd2e0 0x7fb57c0388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c017070 con 0x7fd070161fb0 2024-03-08T08:27:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c0035f0 con 0x7fd070161fb0 2024-03-08T08:27:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0039e0 con 0x7fd070161fb0 2024-03-08T08:27:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb582ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb57c0dd2e0 0x7fb57c0388a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:48882/0 (socket says 172.21.15.105:48882) 2024-03-08T08:27:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f5ab55e5640 1 -- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 msgr2=0x7f5aa8180080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f5ab55e5640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5aa8050e80 0x7f5aa8180080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f5ab55e5640 1 -- 172.21.15.105:0/3508716458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa803a780 msgr2=0x7f5aa80492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f5ab55e5640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa803a780 0x7f5aa80492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f5ab55e5640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa81805c0 con 0x7f5aa80e27a0 2024-03-08T08:27:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd070200f80 con 0x7fd070161fb0 2024-03-08T08:27:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd070201540 con 0x7fd070161fb0 2024-03-08T08:27:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb582ffd640 1 -- 172.21.15.105:0/2372990959 learned_addr learned my addr 172.21.15.105:0/2372990959 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7fae077fe640 1 -- 172.21.15.105:0/962879581 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae1c075120 con 0x7fae1c1645f0 2024-03-08T08:27:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb583fff640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57c0e67c0 0x7fb57c035f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb58a0df640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c051920 con 0x7fb57c0e63e0 2024-03-08T08:27:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7ffbc89c9640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbc41650d0 0x7ffbc4215080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f655effd640 1 -- 172.21.15.105:0/2198400472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65580eb2a0 msgr2=0x7f65580e6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f655effd640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65580eb2a0 0x7f65580e6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.479+0000 7f655effd640 1 -- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6558009190 msgr2=0x7f65580dbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f5aaffff640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5aa803a780 0x7f5aa80492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7ffbc89c9640 1 -- 172.21.15.105:0/4265631909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbc4156ab0 msgr2=0x7ffbc415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7ffbc89c9640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbc4156ab0 0x7ffbc415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7ffbc89c9640 1 -- 172.21.15.105:0/4265631909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4163af0 con 0x7ffbc415eff0 2024-03-08T08:27:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00b4040 con 0x7f16b8038110 2024-03-08T08:27:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb58a0df640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0517a0 con 0x7fb57c0e67c0 2024-03-08T08:27:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb58a0df640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c051aa0 con 0x7fb57c0dd2e0 2024-03-08T08:27:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fb5837fe640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0e63e0 0x7fb57c038360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f655effd640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6558009190 0x7f65580dbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f5ab55e5640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa80e27a0 0x7f5aa80e2c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0069350 tx=0x7f5ab0068790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f655effd640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580e2860 con 0x7f6558048c40 2024-03-08T08:27:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16b8005390 con 0x7f16b8038110 2024-03-08T08:27:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7ffbc89c9640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc415eff0 0x7ffbc4164b90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0000c00 tx=0x7ffbc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f655f7fe640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6558009190 0x7f65580dbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0093600 con 0x7f5aa80e27a0 2024-03-08T08:27:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab00937a0 con 0x7f5aa80e27a0 2024-03-08T08:27:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0093b50 con 0x7f5aa80e27a0 2024-03-08T08:27:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7f5ab6847640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa8186d00 con 0x7f5aa80e27a0 2024-03-08T08:27:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7f5ab6847640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa80487d0 con 0x7f5aa80e27a0 2024-03-08T08:27:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7fb583fff640 1 -- 172.21.15.105:0/2372990959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb57c0dd2e0 msgr2=0x7fb57c0388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7fb583fff640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb57c0dd2e0 0x7fb57c0388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd070075120 con 0x7fd070161fb0 2024-03-08T08:27:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7fb583fff640 1 -- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0e63e0 msgr2=0x7fb57c038360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7fb583fff640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0e63e0 0x7fb57c038360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0042020 con 0x7ffbc415eff0 2024-03-08T08:27:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc002dd70 con 0x7ffbc415eff0 2024-03-08T08:27:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0200049e0 con 0x7fc024165ba0 2024-03-08T08:27:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02406e640 con 0x7fc024165ba0 2024-03-08T08:27:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.480+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02406ec00 con 0x7fc024165ba0 2024-03-08T08:27:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.481+0000 7fb583fff640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c03a460 con 0x7fb57c0e67c0 2024-03-08T08:27:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7f655effd640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6558048c40 0x7f6558007c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6550009fd0 tx=0x7f655000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550015020 con 0x7f6558048c40 2024-03-08T08:27:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6550011850 con 0x7f6558048c40 2024-03-08T08:27:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc4163dd0 con 0x7ffbc415eff0 2024-03-08T08:27:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc4164390 con 0x7ffbc415eff0 2024-03-08T08:27:48.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0039040 con 0x7ffbc415eff0 2024-03-08T08:27:48.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7fb5837fe640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb57c0e63e0 0x7fb57c038360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:48.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0241543d0 con 0x7fc024165ba0 2024-03-08T08:27:48.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc02001a020 con 0x7fc024165ba0 2024-03-08T08:27:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550011c80 con 0x7f6558048c40 2024-03-08T08:27:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5aa8005570 con 0x7f5aa80e27a0 2024-03-08T08:27:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fc019ffb640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc00806b5a0 0x7fc00806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ab009c3f0 con 0x7f5aa80e27a0 2024-03-08T08:27:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.482+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16c00c1070 con 0x7f16b8038110 2024-03-08T08:27:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae0c04b060 con 0x7fae1c1645f0 2024-03-08T08:27:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7f16b6ffd640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16a406b520 0x7f16a406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fb583fff640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57c0e67c0 0x7fb57c035f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb57c055310 tx=0x7fb57000fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fae11ffb640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fae0006b5a0 0x7fae0006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16c0127810 con 0x7f16b8038110 2024-03-08T08:27:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7f16b6ffd640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f16a4073740 0x7f16a4075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 --> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16c006bae0 con 0x7f16a4073740 2024-03-08T08:27:48.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16c0127cf0 con 0x7f16b8038110 2024-03-08T08:27:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7f6565ed4640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65580e8090 con 0x7f6558048c40 2024-03-08T08:27:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7f16c58d9640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16a406b520 0x7f16a406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae0c0abe20 con 0x7fae1c1645f0 2024-03-08T08:27:48.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5700052e0 con 0x7fb57c0e67c0 2024-03-08T08:27:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0200897c0 con 0x7fc024165ba0 2024-03-08T08:27:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fc019ffb640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7fc008073820 0x7fc008075c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 --> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0080762f0 con 0x7fc008073820 2024-03-08T08:27:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc020090050 con 0x7fc024165ba0 2024-03-08T08:27:48.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7f6565ed4640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65580e8650 con 0x7f6558048c40 2024-03-08T08:27:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb570018070 con 0x7fb57c0e67c0 2024-03-08T08:27:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7f16c68db640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f16a4073740 0x7f16a4075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fae11ffb640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7fae000737c0 0x7fae00075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fae21688640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fae0006b5a0 0x7fae0006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 --> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0c001d30 con 0x7fae000737c0 2024-03-08T08:27:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae0c0ae480 con 0x7fae1c1645f0 2024-03-08T08:27:48.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fae21e89640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7fae000737c0 0x7fae00075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7f5aacff9640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5a9406b5a0 0x7f5a9406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ab01050d0 con 0x7f5aa80e27a0 2024-03-08T08:27:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7f5aacff9640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f5a940737c0 0x7f5a94075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ab0097e50 con 0x7f5a940737c0 2024-03-08T08:27:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ab010c050 con 0x7f5aa80e27a0 2024-03-08T08:27:48.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbc0045020 con 0x7ffbc415eff0 2024-03-08T08:27:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7f5aaffff640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5a9406b5a0 0x7f5a9406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7f5aaf7fe640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f5a940737c0 0x7f5a94075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fc028d34640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7fc008073820 0x7fc008075c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd06c01a070 con 0x7fd070161fb0 2024-03-08T08:27:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fd065ffb640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd05406b5a0 0x7fd05406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd06c089650 con 0x7fd070161fb0 2024-03-08T08:27:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fd065ffb640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7fd0540737c0 0x7fd054075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 --> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd06c016ec0 con 0x7fd0540737c0 2024-03-08T08:27:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7fc029535640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc00806b5a0 0x7fc00806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.483+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd06c089a40 con 0x7fd070161fb0 2024-03-08T08:27:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fd07502b640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd05406b5a0 0x7fd05406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000d8c0 con 0x7fb57c0e67c0 2024-03-08T08:27:48.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.484+0000 7fb58a0df640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57c03a740 con 0x7fb57c0e67c0 2024-03-08T08:27:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7fb58a0df640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57c03a150 con 0x7fb57c0e67c0 2024-03-08T08:27:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7fd07482a640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7fd0540737c0 0x7fd054075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65580eabf0 con 0x7f6558048c40 2024-03-08T08:27:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffbc404efc0 con 0x7ffbc415eff0 2024-03-08T08:27:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7ffbb5ffb640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffba806b5d0 0x7ffba806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7fc028d34640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7fc008073820 0x7fc008075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7f16c58d9640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16a406b520 0x7f16a406d9e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f16b8053400 tx=0x7f16bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7f16c68db640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f16a4073740 0x7f16a4075b60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7fae21688640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fae0006b5a0 0x7fae0006da60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae18001560 tx=0x7fae18007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7fae21e89640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7fae000737c0 0x7fae00075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7fd07502b640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd05406b5a0 0x7fd05406da60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd07015df00 tx=0x7fd068030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7ffbc91ca640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffba806b5d0 0x7ffba806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbc00aacc0 con 0x7ffbc415eff0 2024-03-08T08:27:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7ffbb5ffb640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7ffba80737f0 0x7ffba8075c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7f5aaf7fe640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f5a940737c0 0x7f5a94075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7f5aaffff640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5a9406b5a0 0x7f5a9406da60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009630 tx=0x7f5aa4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7fb58a0df640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb57c03ae00 con 0x7fb57c0e67c0 2024-03-08T08:27:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6550018020 con 0x7f6558048c40 2024-03-08T08:27:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 <== osd.6 v2:172.21.15.144:6800/768956787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fae0c001d30 con 0x7fae000737c0 2024-03-08T08:27:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.485+0000 7fc029535640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc00806b5a0 0x7fc00806da60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc01c035000 tx=0x7fc01c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 <== osd.4 v2:172.21.15.144:6801/3800656202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5ab0097e50 con 0x7f5a940737c0 2024-03-08T08:27:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 <== osd.3 v2:172.21.15.105:6808/2646109234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc0080762f0 con 0x7fc008073820 2024-03-08T08:27:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7ffbc99cb640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7ffba80737f0 0x7ffba8075c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7ffbc91ca640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffba806b5d0 0x7ffba806da90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80017e0 tx=0x7ffbb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 --> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffbc0003e90 con 0x7ffba80737f0 2024-03-08T08:27:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7f6547fff640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f654c06b5a0 0x7f654c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.486+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbc00ad270 con 0x7ffbc415eff0 2024-03-08T08:27:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7ffbc99cb640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7ffba80737f0 0x7ffba8075c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f655008aa70 con 0x7f6558048c40 2024-03-08T08:27:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f655f7fe640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f654c06b5a0 0x7f654c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 <== osd.5 v2:172.21.15.144:6816/56982278 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f16c006bae0 con 0x7f16a4073740 2024-03-08T08:27:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f6547fff640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f654c073820 0x7f654c075c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6550011850 con 0x7f654c073820 2024-03-08T08:27:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f655008ae60 con 0x7f6558048c40 2024-03-08T08:27:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f655ffff640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f654c073820 0x7f654c075c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f655f7fe640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f654c06b5a0 0x7f654c06da60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6558055900 tx=0x7f65540090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7f655ffff640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f654c073820 0x7f654c075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb57000da60 con 0x7fb57c0e67c0 2024-03-08T08:27:48.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7fb56ffff640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb56806b520 0x7fb56806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.488+0000 7fb5837fe640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb56806b520 0x7fb56806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.487+0000 7fd07482a640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7fd0540737c0 0x7fd054075be0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.488+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb570089e60 con 0x7fb57c0e67c0 2024-03-08T08:27:48.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.488+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 <== osd.1 v2:172.21.15.105:6824/2989492650 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffbc0003e90 con 0x7ffba80737f0 2024-03-08T08:27:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.488+0000 7fb56ffff640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fb5680736c0 0x7fb568075ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.488+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb57000ae50 con 0x7fb5680736c0 2024-03-08T08:27:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.488+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5700549f0 con 0x7fb57c0e67c0 2024-03-08T08:27:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.488+0000 7fb5837fe640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb56806b520 0x7fb56806d9e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb57c003eb0 tx=0x7fb578009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.488+0000 7fb582ffd640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fb5680736c0 0x7fb568075ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.489+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 <== osd.7 v2:172.21.15.144:6824/1634585826 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd06c016ec0 con 0x7fd0540737c0 2024-03-08T08:27:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.489+0000 7fb582ffd640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fb5680736c0 0x7fb568075ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.489+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 <== osd.0 v2:172.21.15.105:6813/491346206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6550011850 con 0x7f654c073820 2024-03-08T08:27:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.490+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 <== osd.2 v2:172.21.15.105:6800/1086051235 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb57000ae50 con 0x7fb5680736c0 2024-03-08T08:27:48.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.509+0000 7fae077fe640 1 -- 172.21.15.105:0/962879581 --> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae1c09e1b0 con 0x7fae000737c0 2024-03-08T08:27:48.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.510+0000 7fae11ffb640 1 -- 172.21.15.105:0/962879581 <== osd.6 v2:172.21.15.144:6800/768956787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fae1c09e1b0 con 0x7fae000737c0 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.510+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7fae000737c0 msgr2=0x7fae00075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.511+0000 7fae238ec640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7fae000737c0 0x7fae00075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.511+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fae0006b5a0 msgr2=0x7fae0006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.511+0000 7fae238ec640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fae0006b5a0 0x7fae0006da60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae18001560 tx=0x7fae18007990 comp rx=0 tx=0).stop 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.511+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 msgr2=0x7fae1c0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.511+0000 7fae238ec640 1 --2- 172.21.15.105:0/962879581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae1c1645f0 0x7fae1c0c9bd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0090b0 tx=0x7fae0c0046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.511+0000 7fae21e89640 1 -- 172.21.15.105:0/962879581 reap_dead start 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.511+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 shutdown_connections 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.511+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 >> 172.21.15.105:0/962879581 conn(0x7fae1c072110 msgr2=0x7fae1c0b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.515+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 shutdown_connections 2024-03-08T08:27:48.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.517+0000 7fae238ec640 1 -- 172.21.15.105:0/962879581 wait complete. 2024-03-08T08:27:48.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.517+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 --> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd0701654f0 con 0x7fd0540737c0 2024-03-08T08:27:48.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.517+0000 7fd065ffb640 1 -- 172.21.15.105:0/1080563483 <== osd.7 v2:172.21.15.144:6824/1634585826 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd0701654f0 con 0x7fd0540737c0 2024-03-08T08:27:48.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.518+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 --> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5aa8055970 con 0x7f5a940737c0 2024-03-08T08:27:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.518+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7fd0540737c0 msgr2=0x7fd054075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.518+0000 7fd07728f640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7fd0540737c0 0x7fd054075be0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd05406b5a0 msgr2=0x7fd05406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07728f640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd05406b5a0 0x7fd05406da60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd07015df00 tx=0x7fd068030000 comp rx=0 tx=0).stop 2024-03-08T08:27:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 --> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc02404efc0 con 0x7fc008073820 2024-03-08T08:27:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd070161fb0 msgr2=0x7fd0700ab1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07728f640 1 --2- 172.21.15.105:0/1080563483 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd070161fb0 0x7fd0700ab1e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00c3e0 tx=0x7fd06c00c8a0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07582c640 1 -- 172.21.15.105:0/1080563483 reap_dead start 2024-03-08T08:27:48.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7f5aacff9640 1 -- 172.21.15.105:0/3508716458 <== osd.4 v2:172.21.15.144:6801/3800656202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5aa8055970 con 0x7f5a940737c0 2024-03-08T08:27:48.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 shutdown_connections 2024-03-08T08:27:48.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 >> 172.21.15.105:0/1080563483 conn(0x7fd0700b1740 msgr2=0x7fd0700b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 --> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f16b800aa80 con 0x7f16a4073740 2024-03-08T08:27:48.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 shutdown_connections 2024-03-08T08:27:48.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fd07728f640 1 -- 172.21.15.105:0/1080563483 wait complete. 2024-03-08T08:27:48.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.519+0000 7fc019ffb640 1 -- 172.21.15.105:0/706690503 <== osd.3 v2:172.21.15.105:6808/2646109234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc02404efc0 con 0x7fc008073820 2024-03-08T08:27:48.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16b6ffd640 1 -- 172.21.15.105:0/2211731491 <== osd.5 v2:172.21.15.144:6816/56982278 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f16b800aa80 con 0x7f16a4073740 2024-03-08T08:27:48.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f5a940737c0 msgr2=0x7f5a94075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f5a940737c0 0x7f5a94075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5a9406b5a0 msgr2=0x7f5a9406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5a9406b5a0 0x7f5a9406da60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009630 tx=0x7f5aa4008030 comp rx=0 tx=0).stop 2024-03-08T08:27:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa80e27a0 msgr2=0x7f5aa80e2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 --2- 172.21.15.105:0/3508716458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5aa80e27a0 0x7f5aa80e2c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0069350 tx=0x7f5ab0068790 comp rx=0 tx=0).stop 2024-03-08T08:27:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5ab55e5640 1 -- 172.21.15.105:0/3508716458 reap_dead start 2024-03-08T08:27:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7fc008073820 msgr2=0x7fc008075c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7fc02a537640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7fc008073820 0x7fc008075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 shutdown_connections 2024-03-08T08:27:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 >> 172.21.15.105:0/3508716458 conn(0x7f5aa8054ca0 msgr2=0x7f5aa80065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f16a4073740 msgr2=0x7f16a4075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c833e640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f16a4073740 0x7f16a4075b60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc00806b5a0 msgr2=0x7fc00806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7fc02a537640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc00806b5a0 0x7fc00806da60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc01c035000 tx=0x7fc01c030000 comp rx=0 tx=0).stop 2024-03-08T08:27:48.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 shutdown_connections 2024-03-08T08:27:48.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc024165ba0 msgr2=0x7fc024158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7fc02a537640 1 --2- 172.21.15.105:0/706690503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc024165ba0 0x7fc024158e00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c3f0 tx=0x7fc02000c8b0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16a406b520 msgr2=0x7f16a406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c833e640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16a406b520 0x7f16a406d9e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f16b8053400 tx=0x7f16bc008040 comp rx=0 tx=0).stop 2024-03-08T08:27:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16b8038110 msgr2=0x7f16b804e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c833e640 1 --2- 172.21.15.105:0/2211731491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16b8038110 0x7f16b804e0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16c00699b0 tx=0x7f16c0068a30 comp rx=0 tx=0).stop 2024-03-08T08:27:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c68db640 1 -- 172.21.15.105:0/2211731491 reap_dead start 2024-03-08T08:27:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 shutdown_connections 2024-03-08T08:27:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.520+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 >> 172.21.15.105:0/2211731491 conn(0x7f16b8009fb0 msgr2=0x7f16b8036430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7f5a9a7fc640 1 -- 172.21.15.105:0/3508716458 wait complete. 2024-03-08T08:27:48.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7fc029d36640 1 -- 172.21.15.105:0/706690503 reap_dead start 2024-03-08T08:27:48.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 shutdown_connections 2024-03-08T08:27:48.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 >> 172.21.15.105:0/706690503 conn(0x7fc02409cdc0 msgr2=0x7fc0240b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 shutdown_connections 2024-03-08T08:27:48.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7f16c833e640 1 -- 172.21.15.105:0/2211731491 wait complete. 2024-03-08T08:27:48.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 shutdown_connections 2024-03-08T08:27:48.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7fc02a537640 1 -- 172.21.15.105:0/706690503 wait complete. 2024-03-08T08:27:48.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.521+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 --> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffbc406baf0 con 0x7ffba80737f0 2024-03-08T08:27:48.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.522+0000 7ffbb5ffb640 1 -- 172.21.15.105:0/4265631909 <== osd.1 v2:172.21.15.105:6824/2989492650 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffbc406baf0 con 0x7ffba80737f0 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.522+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7ffba80737f0 msgr2=0x7ffba8075c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.522+0000 7ffbca1cc640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7ffba80737f0 0x7ffba8075c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffba806b5d0 msgr2=0x7ffba806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbca1cc640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffba806b5d0 0x7ffba806da90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80017e0 tx=0x7ffbb8009040 comp rx=0 tx=0).stop 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc415eff0 msgr2=0x7ffbc4164b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbca1cc640 1 --2- 172.21.15.105:0/4265631909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc415eff0 0x7ffbc4164b90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0000c00 tx=0x7ffbc0004730 comp rx=0 tx=0).stop 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbc99cb640 1 -- 172.21.15.105:0/4265631909 reap_dead start 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 shutdown_connections 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 >> 172.21.15.105:0/4265631909 conn(0x7ffbc406ac80 msgr2=0x7ffbc4161930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 shutdown_connections 2024-03-08T08:27:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7ffbca1cc640 1 -- 172.21.15.105:0/4265631909 wait complete. 2024-03-08T08:27:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.523+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 --> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65580e8850 con 0x7f654c073820 2024-03-08T08:27:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.524+0000 7fb58a0df640 1 -- 172.21.15.105:0/2372990959 --> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb57c005a60 con 0x7fb5680736c0 2024-03-08T08:27:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.524+0000 7f6547fff640 1 -- 172.21.15.105:0/2198400472 <== osd.0 v2:172.21.15.105:6813/491346206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f65580e8850 con 0x7f654c073820 2024-03-08T08:27:48.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.525+0000 7fb56ffff640 1 -- 172.21.15.105:0/2372990959 <== osd.2 v2:172.21.15.105:6800/1086051235 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb57c005a60 con 0x7fb5680736c0 2024-03-08T08:27:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.525+0000 7fb56dffb640 1 -- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fb5680736c0 msgr2=0x7fb568075ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.525+0000 7fb56dffb640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fb5680736c0 0x7fb568075ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7fb56dffb640 1 -- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb56806b520 msgr2=0x7fb56806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7fb56dffb640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb56806b520 0x7fb56806d9e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb57c003eb0 tx=0x7fb578009190 comp rx=0 tx=0).stop 2024-03-08T08:27:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7fb56dffb640 1 -- 172.21.15.105:0/2372990959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57c0e67c0 msgr2=0x7fb57c035f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7fb56dffb640 1 --2- 172.21.15.105:0/2372990959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb57c0e67c0 0x7fb57c035f10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb57c055310 tx=0x7fb57000fd40 comp rx=0 tx=0).stop 2024-03-08T08:27:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7fb583fff640 1 -- 172.21.15.105:0/2372990959 reap_dead start 2024-03-08T08:27:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f654c073820 msgr2=0x7f654c075c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7f6545ffb640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f654c073820 0x7f654c075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7fb56dffb640 1 -- 172.21.15.105:0/2372990959 shutdown_connections 2024-03-08T08:27:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7fb56dffb640 1 -- 172.21.15.105:0/2372990959 >> 172.21.15.105:0/2372990959 conn(0x7fb57c055380 msgr2=0x7fb57c049760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.526+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f654c06b5a0 msgr2=0x7f654c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7f6545ffb640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f654c06b5a0 0x7f654c06da60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6558055900 tx=0x7f65540090e0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6558048c40 msgr2=0x7f6558007c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7f6545ffb640 1 --2- 172.21.15.105:0/2198400472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6558048c40 0x7f6558007c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6550009fd0 tx=0x7f655000b870 comp rx=0 tx=0).stop 2024-03-08T08:27:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7f655ffff640 1 -- 172.21.15.105:0/2198400472 reap_dead start 2024-03-08T08:27:48.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 shutdown_connections 2024-03-08T08:27:48.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 >> 172.21.15.105:0/2198400472 conn(0x7f655801ef40 msgr2=0x7f655803b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7fb56dffb640 1 -- 172.21.15.105:0/2372990959 shutdown_connections 2024-03-08T08:27:48.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7fb56dffb640 1 -- 172.21.15.105:0/2372990959 wait complete. 2024-03-08T08:27:48.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 shutdown_connections 2024-03-08T08:27:48.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.527+0000 7f6545ffb640 1 -- 172.21.15.105:0/2198400472 wait complete. 2024-03-08T08:27:48.585 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:48.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-08T08:27:48.586 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2024-03-08T08:27:48.587 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-08T08:27:48.587 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-03-08T08:27:48.588 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-08T08:27:48.588 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2024-03-08T08:27:48.589 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-08T08:27:48.589 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:48.590 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-08T08:27:48.590 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:48.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-08T08:27:48.592 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:48.592 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-08T08:27:48.593 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:48.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-08T08:27:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.792+0000 7fa7002ae640 1 Processor -- start 2024-03-08T08:27:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.792+0000 7fa7002ae640 1 -- start start 2024-03-08T08:27:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.793+0000 7fa7002ae640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6f816a6d0 0x7fa6f80716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.793+0000 7fa7002ae640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa6f816aab0 0x7fa6f8071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.793+0000 7fa7002ae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f80b49a0 0x7fa6f80b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.793+0000 7fa7002ae640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80b3be0 con 0x7fa6f816a6d0 2024-03-08T08:27:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.793+0000 7fa7002ae640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80b3ec0 con 0x7fa6f816aab0 2024-03-08T08:27:48.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.793+0000 7fa7002ae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b41a0 con 0x7fa6f80b49a0 2024-03-08T08:27:48.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.794+0000 7fa7f57b5640 1 Processor -- start 2024-03-08T08:27:48.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.794+0000 7fa7f57b5640 1 -- start start 2024-03-08T08:27:48.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.794+0000 7fa6fe04a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6f816a6d0 0x7fa6f80716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.794+0000 7fa6fe04a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6f816a6d0 0x7fa6f80716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59434/0 (socket says 172.21.15.105:59434) 2024-03-08T08:27:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.794+0000 7fa6fe04a640 1 -- 172.21.15.105:0/2755281371 learned_addr learned my addr 172.21.15.105:0/2755281371 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.794+0000 7fa6fd849640 1 --2- 172.21.15.105:0/2755281371 >> v2:172.21.15.144:3300/0 conn(0x7fa6f816aab0 0x7fa6f8071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.794+0000 7fa6fe84b640 1 --2- 172.21.15.105:0/2755281371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f80b49a0 0x7fa6f80b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7f57b5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa7f0162480 0x7fa7f00b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7f57b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 0x7fa7f00b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7f57b5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f00b4df0 con 0x7fa7f00b4770 2024-03-08T08:27:48.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7f57b5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f006f140 con 0x7fa7f0162480 2024-03-08T08:27:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7f57b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0165f70 con 0x7fa7f00cf7b0 2024-03-08T08:27:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa6fe04a640 1 -- 172.21.15.105:0/2755281371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f80b49a0 msgr2=0x7fa6f80b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa6fe04a640 1 --2- 172.21.15.105:0/2755281371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f80b49a0 0x7fa6f80b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa6fe04a640 1 -- 172.21.15.105:0/2755281371 >> v2:172.21.15.144:3300/0 conn(0x7fa6f816aab0 msgr2=0x7fa6f8071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa6fe04a640 1 --2- 172.21.15.105:0/2755281371 >> v2:172.21.15.144:3300/0 conn(0x7fa6f816aab0 0x7fa6f8071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa6fe04a640 1 -- 172.21.15.105:0/2755281371 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80b4480 con 0x7fa6f816a6d0 2024-03-08T08:27:48.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7ee7fc640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa7f00b4770 0x7fa7f00cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:40862/0 (socket says 172.21.15.105:40862) 2024-03-08T08:27:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7ee7fc640 1 -- 172.21.15.105:0/3731023150 learned_addr learned my addr 172.21.15.105:0/3731023150 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7eeffd640 1 --2- 172.21.15.105:0/3731023150 >> v2:172.21.15.105:3300/0 conn(0x7fa7f0162480 0x7fa7f00b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7fa7ef7fe640 1 --2- 172.21.15.105:0/3731023150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 0x7fa7f00b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7fa7edffb640 1 -- 172.21.15.105:0/3731023150 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218759062 0 0) 0x7fa7f00b4df0 con 0x7fa7f00b4770 2024-03-08T08:27:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7fa7edffb640 1 -- 172.21.15.105:0/3731023150 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7d8003520 con 0x7fa7f00b4770 2024-03-08T08:27:48.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.795+0000 7f7db61e5640 1 Processor -- start 2024-03-08T08:27:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7fe60271e640 1 Processor -- start 2024-03-08T08:27:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7fa7ef7fe640 1 -- 172.21.15.105:0/3731023150 >> v2:172.21.15.105:3300/0 conn(0x7fa7f0162480 msgr2=0x7fa7f00b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7fa7ef7fe640 1 --2- 172.21.15.105:0/3731023150 >> v2:172.21.15.105:3300/0 conn(0x7fa7f0162480 0x7fa7f00b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7fa7ef7fe640 1 -- 172.21.15.105:0/3731023150 >> v1:172.21.15.144:6789/0 conn(0x7fa7f00b4770 legacy=0x7fa7f00cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7fa7ef7fe640 1 -- 172.21.15.105:0/3731023150 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f0166250 con 0x7fa7f00cf7b0 2024-03-08T08:27:48.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7fe60271e640 1 -- start start 2024-03-08T08:27:48.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.796+0000 7f7db61e5640 1 -- start start 2024-03-08T08:27:48.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db61e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db0162200 0x7f7db00b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fa6fe04a640 1 --2- 172.21.15.105:0/2755281371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f816a6d0 0x7fa6f80716b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6f816e3b0 tx=0x7fa6f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d60ffefa10c6d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db61e5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7db006c3a0 0x7f7db00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fd0cabae640 1 Processor -- start 2024-03-08T08:27:48.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fd0cabae640 1 -- start start 2024-03-08T08:27:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7ffb7969d640 1 Processor -- start 2024-03-08T08:27:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fe60271e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5fc15aad0 0x7fe5fc15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fa7ef7fe640 1 --2- 172.21.15.105:0/3731023150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 0x7fa7f00b2cc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009d90 tx=0x7fa7e802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e312a53a2df42bd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fe60271e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7ffb7969d640 1 -- start start 2024-03-08T08:27:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db61e5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db0165d10 con 0x7f7db00cf360 2024-03-08T08:27:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db61e5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00cfd00 con 0x7f7db006c3a0 2024-03-08T08:27:48.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db61e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b40f0 con 0x7f7db0162200 2024-03-08T08:27:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db49e2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7db00cf360 0x7f7db006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40302/0 (socket says 172.21.15.105:40302) 2024-03-08T08:27:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db49e2640 1 -- 172.21.15.105:0/4294632419 learned_addr learned my addr 172.21.15.105:0/4294632419 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fe60271e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5fc06ec20 0x7fe5fc06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fe60271e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5fc15ebb0 con 0x7fe5fc06ec20 2024-03-08T08:27:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fa7edffb640 1 -- 172.21.15.105:0/3731023150 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8039070 con 0x7fa7f00cf7b0 2024-03-08T08:27:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fa7edffb640 1 -- 172.21.15.105:0/3731023150 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e802cc50 con 0x7fa7f00cf7b0 2024-03-08T08:27:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db59e4640 1 --2- 172.21.15.105:0/4294632419 >> v2:172.21.15.144:3300/0 conn(0x7f7db006c3a0 0x7f7db00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7f7db51e3640 1 --2- 172.21.15.105:0/4294632419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db0162200 0x7f7db00b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fe60271e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5fc15ee90 con 0x7fe5fc15aad0 2024-03-08T08:27:48.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.797+0000 7fe60271e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc15f170 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fa7edffb640 1 -- 172.21.15.105:0/3731023150 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8034440 con 0x7fa7f00cf7b0 2024-03-08T08:27:48.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fe600cbb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5fc06ec20 0x7fe5fc06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fe600cbb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe5fc06ec20 0x7fe5fc06d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59450/0 (socket says 172.21.15.105:59450) 2024-03-08T08:27:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fe600cbb640 1 -- 172.21.15.105:0/1805690662 learned_addr learned my addr 172.21.15.105:0/1805690662 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fe5fbfff640 1 --2- 172.21.15.105:0/1805690662 >> v2:172.21.15.144:3300/0 conn(0x7fe5fc15aad0 0x7fe5fc15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fa6fd048640 1 -- 172.21.15.105:0/2755281371 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f002cb00 con 0x7fa6f816a6d0 2024-03-08T08:27:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fa6fd048640 1 -- 172.21.15.105:0/2755281371 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f002cca0 con 0x7fa6f816a6d0 2024-03-08T08:27:48.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fe5fb7fe640 1 --2- 172.21.15.105:0/1805690662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fd0cabae640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd0c40716b0 0x7fd0c4071a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fd0cabae640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c4071fd0 con 0x7fd0c4145a00 2024-03-08T08:27:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fd0cabae640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c4162680 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fd0cabae640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c41455c0 con 0x7fd0c40716b0 2024-03-08T08:27:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7ffb7969d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c007400 0x7ffb6c008c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7ffb7969d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c0095a0 con 0x7ffb6c0e7890 2024-03-08T08:27:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7ffb7969d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c050780 con 0x7ffb6c034850 2024-03-08T08:27:48.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7ffb7969d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c007840 con 0x7ffb6c007400 2024-03-08T08:27:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7f57b5640 1 -- 172.21.15.105:0/3731023150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 msgr2=0x7fa7f00b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7f57b5640 1 --2- 172.21.15.105:0/3731023150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 0x7fa7f00b2cc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009d90 tx=0x7fa7e802bea0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7f57b5640 1 -- 172.21.15.105:0/3731023150 shutdown_connections 2024-03-08T08:27:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7f57b5640 1 --2- 172.21.15.105:0/3731023150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 0x7fa7f00b2cc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7f57b5640 1 --2- 172.21.15.105:0/3731023150 >> v2:172.21.15.105:3300/0 conn(0x7fa7f0162480 0x7fa7f00b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7f57b5640 1 -- 172.21.15.105:0/3731023150 >> 172.21.15.105:0/3731023150 conn(0x7fa7f00cfd10 msgr2=0x7fa7f00d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fd0c3fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd0c40cf4e0 0x7fd0c406df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:40864/0 (socket says 172.21.15.105:40864) 2024-03-08T08:27:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fd0c3fff640 1 -- 172.21.15.105:0/435544360 learned_addr learned my addr 172.21.15.105:0/435544360 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fd0c894a640 1 --2- 172.21.15.105:0/435544360 >> v2:172.21.15.105:3300/0 conn(0x7fd0c40716b0 0x7fd0c4071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7f9a10a22640 1 Processor -- start 2024-03-08T08:27:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7f9a10a22640 1 -- start start 2024-03-08T08:27:48.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7fe600cbb640 1 -- 172.21.15.105:0/1805690662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 msgr2=0x7fe5fc06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa6fd048640 1 -- 172.21.15.105:0/2755281371 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0034610 con 0x7fa6f816a6d0 2024-03-08T08:27:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7ffb73fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffb6c0e7890 0x7ffb6c051540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40322/0 (socket says 172.21.15.105:40322) 2024-03-08T08:27:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7ffb73fff640 1 -- 172.21.15.105:0/800679204 learned_addr learned my addr 172.21.15.105:0/800679204 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7f7db59e4640 1 -- 172.21.15.105:0/4294632419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db0162200 msgr2=0x7f7db00b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7f7db59e4640 1 --2- 172.21.15.105:0/4294632419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db0162200 0x7f7db00b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7f7db59e4640 1 -- 172.21.15.105:0/4294632419 >> v1:172.21.15.105:6789/0 conn(0x7f7db00cf360 legacy=0x7f7db006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.798+0000 7f7db59e4640 1 -- 172.21.15.105:0/4294632419 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db01625e0 con 0x7f7db006c3a0 2024-03-08T08:27:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7002ae640 1 -- 172.21.15.105:0/2755281371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f816a6d0 msgr2=0x7fa6f80716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7002ae640 1 --2- 172.21.15.105:0/2755281371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f816a6d0 0x7fa6f80716b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6f816e3b0 tx=0x7fa6f002bf60 comp rx=0 tx=0).stop 2024-03-08T08:27:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fe600cbb640 1 --2- 172.21.15.105:0/1805690662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc06e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7ffb737fe640 1 --2- 172.21.15.105:0/800679204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c007400 0x7ffb6c008c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fe600cbb640 1 -- 172.21.15.105:0/1805690662 >> v2:172.21.15.144:3300/0 conn(0x7fe5fc15aad0 msgr2=0x7fe5fc15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fe72bfff640 1 Processor -- start 2024-03-08T08:27:48.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fe72bfff640 1 -- start start 2024-03-08T08:27:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fe600cbb640 1 --2- 172.21.15.105:0/1805690662 >> v2:172.21.15.144:3300/0 conn(0x7fe5fc15aad0 0x7fe5fc15e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fe600cbb640 1 -- 172.21.15.105:0/1805690662 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc15f450 con 0x7fe5fc06ec20 2024-03-08T08:27:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fe5fb7fe640 1 --2- 172.21.15.105:0/1805690662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7ffb72ffd640 1 -- 172.21.15.105:0/800679204 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449080756 0 0) 0x7ffb6c0095a0 con 0x7ffb6c0e7890 2024-03-08T08:27:48.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7ffb72ffd640 1 -- 172.21.15.105:0/800679204 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb64003520 con 0x7ffb6c0e7890 2024-03-08T08:27:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7002ae640 1 -- 172.21.15.105:0/2755281371 shutdown_connections 2024-03-08T08:27:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7002ae640 1 --2- 172.21.15.105:0/2755281371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f80b49a0 0x7fa6f80b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7002ae640 1 --2- 172.21.15.105:0/2755281371 >> v2:172.21.15.144:3300/0 conn(0x7fa6f816aab0 0x7fa6f8071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7002ae640 1 --2- 172.21.15.105:0/2755281371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f816a6d0 0x7fa6f80716b0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6f816e3b0 tx=0x7fa6f002bf60 comp rx=0 tx=0).stop 2024-03-08T08:27:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fa7002ae640 1 -- 172.21.15.105:0/2755281371 >> 172.21.15.105:0/2755281371 conn(0x7fa6f806ac80 msgr2=0x7fa6f806b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fe5fbfff640 1 --2- 172.21.15.105:0/1805690662 >> v2:172.21.15.144:3300/0 conn(0x7fe5fc15aad0 0x7fe5fc15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:48.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fa7f57b5640 1 -- 172.21.15.105:0/3731023150 shutdown_connections 2024-03-08T08:27:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f7db59e4640 1 --2- 172.21.15.105:0/4294632419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00cf7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009d90 tx=0x7f7da802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5067c7d05940ccd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fd0c37fe640 1 -- 172.21.15.105:0/435544360 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808118794 0 0) 0x7fd0c4162680 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fa7002ae640 1 -- 172.21.15.105:0/2755281371 shutdown_connections 2024-03-08T08:27:48.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fa7002ae640 1 -- 172.21.15.105:0/2755281371 wait complete. 2024-03-08T08:27:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fd0c894a640 1 -- 172.21.15.105:0/435544360 >> v1:172.21.15.144:6789/0 conn(0x7fd0c40cf4e0 legacy=0x7fd0c406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fd0c894a640 1 -- 172.21.15.105:0/435544360 >> v1:172.21.15.105:6789/0 conn(0x7fd0c4145a00 legacy=0x7fd0c4145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.799+0000 7fd0c894a640 1 -- 172.21.15.105:0/435544360 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c4162940 con 0x7fd0c40716b0 2024-03-08T08:27:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7ffb737fe640 1 -- 172.21.15.105:0/800679204 >> v1:172.21.15.144:6789/0 conn(0x7ffb6c034850 legacy=0x7ffb6c053790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7ffb737fe640 1 -- 172.21.15.105:0/800679204 >> v1:172.21.15.105:6789/0 conn(0x7ffb6c0e7890 legacy=0x7ffb6c051540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7ffb737fe640 1 -- 172.21.15.105:0/800679204 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c050a60 con 0x7ffb6c007400 2024-03-08T08:27:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f7da7fff640 1 -- 172.21.15.105:0/4294632419 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8039070 con 0x7f7db006c3a0 2024-03-08T08:27:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f7da7fff640 1 -- 172.21.15.105:0/4294632419 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da802cc50 con 0x7f7db006c3a0 2024-03-08T08:27:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f7da7fff640 1 -- 172.21.15.105:0/4294632419 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8034440 con 0x7f7db006c3a0 2024-03-08T08:27:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fe600cbb640 1 --2- 172.21.15.105:0/1805690662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc06ec20 0x7fe5fc06d4c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc06a520 tx=0x7fe5f402c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcbb3adc5d665a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a10a22640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0b38f0 0x7f9a0c06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a10a22640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a0c06b1c0 0x7f9a0c15f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a10a22640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0b50d0 con 0x7f9a0c0afd40 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a10a22640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c06b630 con 0x7f9a0c0b38f0 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a10a22640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c06ee10 con 0x7f9a0c06b1c0 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a0affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0b38f0 0x7f9a0c06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a0affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0b38f0 0x7f9a0c06ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59470/0 (socket says 172.21.15.105:59470) 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a0bfff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a0c06b1c0 0x7f9a0c15f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7f9a0affd640 1 -- 172.21.15.105:0/4028279755 learned_addr learned my addr 172.21.15.105:0/4028279755 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fe72bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe72c1441a0 0x7fe72c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fe72bfff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe72c162010 0x7fe72c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fe72bfff640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c0adfc0 con 0x7fe72c06dc30 2024-03-08T08:27:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fd0c894a640 1 --2- 172.21.15.105:0/435544360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 0x7fd0c4071a90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009d90 tx=0x7fd0b402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd338c9431d9c1c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fe72bfff640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c144580 con 0x7fe72c1441a0 2024-03-08T08:27:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fe72bfff640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c162650 con 0x7fe72c162010 2024-03-08T08:27:48.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe5faffd640 1 -- 172.21.15.105:0/1805690662 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f402cae0 con 0x7fe5fc06ec20 2024-03-08T08:27:48.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe72b7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe72c162010 0x7fe72c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe72b7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe72c162010 0x7fe72c0ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59722/0 (socket says 172.21.15.105:59722) 2024-03-08T08:27:48.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe72b7fe640 1 -- 172.21.15.105:0/215681899 learned_addr learned my addr 172.21.15.105:0/215681899 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe5faffd640 1 -- 172.21.15.105:0/1805690662 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f402cc80 con 0x7fe5fc06ec20 2024-03-08T08:27:48.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7f9a0bfff640 1 -- 172.21.15.105:0/4028279755 >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0b38f0 msgr2=0x7f9a0c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7f7db61e5640 1 -- 172.21.15.105:0/4294632419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 msgr2=0x7f7db00cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7f7db61e5640 1 --2- 172.21.15.105:0/4294632419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00cf7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009d90 tx=0x7f7da802bea0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7ffb737fe640 1 --2- 172.21.15.105:0/800679204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c007400 0x7ffb6c008c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009780 tx=0x7ffb6002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763944767a982f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.800+0000 7fa7f57b5640 1 -- 172.21.15.105:0/3731023150 wait complete. 2024-03-08T08:27:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7f9a0bfff640 1 --2- 172.21.15.105:0/4028279755 >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0b38f0 0x7f9a0c06ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7f9a0bfff640 1 -- 172.21.15.105:0/4028279755 >> v1:172.21.15.144:6789/0 conn(0x7f9a0c0afd40 legacy=0x7f9a0c163020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe72affd640 1 --2- 172.21.15.105:0/215681899 >> v2:172.21.15.105:3300/0 conn(0x7fe72c1441a0 0x7fe72c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7f9a0bfff640 1 -- 172.21.15.105:0/4028279755 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c06f0f0 con 0x7f9a0c06b1c0 2024-03-08T08:27:48.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7f9a0affd640 1 --2- 172.21.15.105:0/4028279755 >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0b38f0 0x7f9a0c06ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fa7002ae640 1 Processor -- start 2024-03-08T08:27:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe72b7fe640 1 -- 172.21.15.105:0/215681899 >> v2:172.21.15.105:3300/0 conn(0x7fe72c1441a0 msgr2=0x7fe72c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe72b7fe640 1 --2- 172.21.15.105:0/215681899 >> v2:172.21.15.105:3300/0 conn(0x7fe72c1441a0 0x7fe72c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe72b7fe640 1 -- 172.21.15.105:0/215681899 >> v1:172.21.15.105:6789/0 conn(0x7fe72c06dc30 legacy=0x7fe72c161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:48.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.801+0000 7fe72b7fe640 1 -- 172.21.15.105:0/215681899 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c0ae280 con 0x7fe72c162010 2024-03-08T08:27:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fa7002ae640 1 -- start start 2024-03-08T08:27:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7ffb72ffd640 1 -- 172.21.15.105:0/800679204 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60039070 con 0x7ffb6c007400 2024-03-08T08:27:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7ffb72ffd640 1 -- 172.21.15.105:0/800679204 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb60030070 con 0x7ffb6c007400 2024-03-08T08:27:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fd0c37fe640 1 -- 172.21.15.105:0/435544360 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4039070 con 0x7fd0c40716b0 2024-03-08T08:27:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fd0c37fe640 1 -- 172.21.15.105:0/435544360 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b402cc60 con 0x7fd0c40716b0 2024-03-08T08:27:48.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fd0c37fe640 1 -- 172.21.15.105:0/435544360 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4039070 con 0x7fd0c40716b0 2024-03-08T08:27:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7ffb72ffd640 1 -- 172.21.15.105:0/800679204 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600318f0 con 0x7ffb6c007400 2024-03-08T08:27:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe5faffd640 1 -- 172.21.15.105:0/1805690662 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f402cae0 con 0x7fe5fc06ec20 2024-03-08T08:27:48.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fa7f57b5640 1 Processor -- start 2024-03-08T08:27:48.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7f7db61e5640 1 -- 172.21.15.105:0/4294632419 shutdown_connections 2024-03-08T08:27:48.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7f7db61e5640 1 --2- 172.21.15.105:0/4294632419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00cf7c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7f7db61e5640 1 --2- 172.21.15.105:0/4294632419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db0162200 0x7f7db00b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7f7db61e5640 1 -- 172.21.15.105:0/4294632419 >> 172.21.15.105:0/4294632419 conn(0x7f7db00cc950 msgr2=0x7f7db00ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 -- 172.21.15.105:0/1805690662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc06ec20 msgr2=0x7fe5fc06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 --2- 172.21.15.105:0/1805690662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc06ec20 0x7fe5fc06d4c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc06a520 tx=0x7fe5f402c140 comp rx=0 tx=0).stop 2024-03-08T08:27:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 -- 172.21.15.105:0/1805690662 shutdown_connections 2024-03-08T08:27:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 --2- 172.21.15.105:0/1805690662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc06ec20 0x7fe5fc06d4c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 --2- 172.21.15.105:0/1805690662 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 --2- 172.21.15.105:0/1805690662 >> v2:172.21.15.144:3300/0 conn(0x7fe5fc15aad0 0x7fe5fc15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 -- 172.21.15.105:0/1805690662 >> 172.21.15.105:0/1805690662 conn(0x7fe5fc06bd90 msgr2=0x7fe5fc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7f9a0bfff640 1 --2- 172.21.15.105:0/4028279755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c06b1c0 0x7f9a0c15f1e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c166c30 tx=0x7f99fc02b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc9152a99f5255e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 -- 172.21.15.105:0/1805690662 shutdown_connections 2024-03-08T08:27:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fe60271e640 1 -- 172.21.15.105:0/1805690662 wait complete. 2024-03-08T08:27:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fa7f57b5640 1 -- start start 2024-03-08T08:27:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7f7db61e5640 1 -- 172.21.15.105:0/4294632419 shutdown_connections 2024-03-08T08:27:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7f9a0a7fc640 1 -- 172.21.15.105:0/4028279755 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc037070 con 0x7f9a0c06b1c0 2024-03-08T08:27:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7f9a0a7fc640 1 -- 172.21.15.105:0/4028279755 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc030710 con 0x7f9a0c06b1c0 2024-03-08T08:27:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.802+0000 7fa7002ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6f80b49a0 0x7fa6f8176eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7002ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f816aab0 0x7fa6f81590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7002ae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f816ca70 0x7fa6f816d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7002ae640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b3450 con 0x7fa6f816aab0 2024-03-08T08:27:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7002ae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b32d0 con 0x7fa6f80b49a0 2024-03-08T08:27:48.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7002ae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b35d0 con 0x7fa6f816ca70 2024-03-08T08:27:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7f9a0a7fc640 1 -- 172.21.15.105:0/4028279755 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc030ae0 con 0x7f9a0c06b1c0 2024-03-08T08:27:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe04a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6f80b49a0 0x7fa6f8176eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe04a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6f80b49a0 0x7fa6f8176eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59726/0 (socket says 172.21.15.105:59726) 2024-03-08T08:27:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe04a640 1 -- 172.21.15.105:0/3829372716 learned_addr learned my addr 172.21.15.105:0/3829372716 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe84b640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f816ca70 0x7fa6f816d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7ffb7969d640 1 -- 172.21.15.105:0/800679204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c007400 msgr2=0x7ffb6c008c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7ffb7969d640 1 --2- 172.21.15.105:0/800679204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c007400 0x7ffb6c008c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009780 tx=0x7ffb6002b770 comp rx=0 tx=0).stop 2024-03-08T08:27:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7ffb7969d640 1 -- 172.21.15.105:0/800679204 shutdown_connections 2024-03-08T08:27:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7ffb7969d640 1 --2- 172.21.15.105:0/800679204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c007400 0x7ffb6c008c50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7ffb7969d640 1 -- 172.21.15.105:0/800679204 >> 172.21.15.105:0/800679204 conn(0x7ffb6c055bc0 msgr2=0x7ffb6c055fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7ffb7969d640 1 -- 172.21.15.105:0/800679204 shutdown_connections 2024-03-08T08:27:48.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fe72b7fe640 1 --2- 172.21.15.105:0/215681899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe72c162010 0x7fe72c0ce750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe72c1617d0 tx=0x7fe72402be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87bec9085e078b18 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7ffb7969d640 1 -- 172.21.15.105:0/800679204 wait complete. 2024-03-08T08:27:48.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fd0cabae640 1 -- 172.21.15.105:0/435544360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 msgr2=0x7fd0c4071a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fd0cabae640 1 --2- 172.21.15.105:0/435544360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 0x7fd0c4071a90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009d90 tx=0x7fd0b402beb0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe84b640 1 -- 172.21.15.105:0/3829372716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6f80b49a0 msgr2=0x7fa6f8176eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe84b640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6f80b49a0 0x7fa6f8176eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe84b640 1 -- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f816aab0 msgr2=0x7fa6f81590d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe84b640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f816aab0 0x7fa6f81590d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa6fe84b640 1 -- 172.21.15.105:0/3829372716 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8159610 con 0x7fa6f816ca70 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fe729ffb640 1 -- 172.21.15.105:0/215681899 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72402c9f0 con 0x7fe72c162010 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fe729ffb640 1 -- 172.21.15.105:0/215681899 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72402cb90 con 0x7fe72c162010 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fe729ffb640 1 -- 172.21.15.105:0/215681899 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7240343f0 con 0x7fe72c162010 2024-03-08T08:27:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7f57b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f00b4770 0x7fa7f0152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7f57b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 0x7fa7f01605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7f57b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f0162480 0x7fa7f0160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7f57b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0166d30 con 0x7fa7f0162480 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7f57b5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0166810 con 0x7fa7f00b4770 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7f57b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0071b30 con 0x7fa7f00cf7b0 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f0162480 0x7fa7f0160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f0162480 0x7fa7f0160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59494/0 (socket says 172.21.15.105:59494) 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fa7ef7fe640 1 -- 172.21.15.105:0/1906123947 learned_addr learned my addr 172.21.15.105:0/1906123947 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fd0cabae640 1 -- 172.21.15.105:0/435544360 shutdown_connections 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fd0cabae640 1 --2- 172.21.15.105:0/435544360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 0x7fd0c4071a90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7fd0cabae640 1 -- 172.21.15.105:0/435544360 >> 172.21.15.105:0/435544360 conn(0x7fd0c406ac80 msgr2=0x7fd0c406b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7f9a10a22640 1 -- 172.21.15.105:0/4028279755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c06b1c0 msgr2=0x7f9a0c15f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7f9a10a22640 1 --2- 172.21.15.105:0/4028279755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c06b1c0 0x7f9a0c15f1e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c166c30 tx=0x7f99fc02b710 comp rx=0 tx=0).stop 2024-03-08T08:27:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa6fe84b640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f816ca70 0x7fa6f816d290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800d530 tx=0x7fa6e800d9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7f9a10a22640 1 -- 172.21.15.105:0/4028279755 shutdown_connections 2024-03-08T08:27:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7f9a10a22640 1 --2- 172.21.15.105:0/4028279755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c06b1c0 0x7f9a0c15f1e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7f9a10a22640 1 --2- 172.21.15.105:0/4028279755 >> v2:172.21.15.105:3300/0 conn(0x7f9a0c0b38f0 0x7f9a0c06ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7f9a10a22640 1 -- 172.21.15.105:0/4028279755 >> 172.21.15.105:0/4028279755 conn(0x7f9a0c06bd90 msgr2=0x7f9a0c06c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.803+0000 7f7db61e5640 1 -- 172.21.15.105:0/4294632419 wait complete. 2024-03-08T08:27:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7f9a10a22640 1 -- 172.21.15.105:0/4028279755 shutdown_connections 2024-03-08T08:27:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa7ef7fe640 1 -- 172.21.15.105:0/1906123947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 msgr2=0x7fa7f01605a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T08:27:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa7ef7fe640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 0x7fa7f01605a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa7ef7fe640 1 -- 172.21.15.105:0/1906123947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f00b4770 msgr2=0x7fa7f0152040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T08:27:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa7ef7fe640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f00b4770 0x7fa7f0152040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa7ef7fe640 1 -- 172.21.15.105:0/1906123947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f0163920 con 0x7fa7f0162480 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa7ef7fe640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f0162480 0x7fa7f0160ae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009eb0 tx=0x7fa7e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fe72bfff640 1 -- 172.21.15.105:0/215681899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe72c162010 msgr2=0x7fe72c0ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fe72bfff640 1 --2- 172.21.15.105:0/215681899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe72c162010 0x7fe72c0ce750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe72c1617d0 tx=0x7fe72402be50 comp rx=0 tx=0).stop 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fe72bfff640 1 -- 172.21.15.105:0/215681899 shutdown_connections 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fe72bfff640 1 --2- 172.21.15.105:0/215681899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe72c162010 0x7fe72c0ce750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fe72bfff640 1 --2- 172.21.15.105:0/215681899 >> v2:172.21.15.105:3300/0 conn(0x7fe72c1441a0 0x7fe72c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fe72bfff640 1 -- 172.21.15.105:0/215681899 >> 172.21.15.105:0/215681899 conn(0x7fe72c06ac80 msgr2=0x7fe72c06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:48.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fd0cabae640 1 -- 172.21.15.105:0/435544360 shutdown_connections 2024-03-08T08:27:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fd0cabae640 1 -- 172.21.15.105:0/435544360 wait complete. 2024-03-08T08:27:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7f7db61e5640 1 Processor -- start 2024-03-08T08:27:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7ffb7969d640 1 Processor -- start 2024-03-08T08:27:48.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7f9a10a22640 1 -- 172.21.15.105:0/4028279755 wait complete. 2024-03-08T08:27:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa6eeffd640 1 -- 172.21.15.105:0/3829372716 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80125f0 con 0x7fa6f816ca70 2024-03-08T08:27:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fa6eeffd640 1 -- 172.21.15.105:0/3829372716 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8019070 con 0x7fa6f816ca70 2024-03-08T08:27:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7fa6eeffd640 1 -- 172.21.15.105:0/3829372716 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80129b0 con 0x7fa6f816ca70 2024-03-08T08:27:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7ffb7969d640 1 -- start start 2024-03-08T08:27:48.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7fa7002ae640 1 -- 172.21.15.105:0/3829372716 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f815dd00 con 0x7fa6f816ca70 2024-03-08T08:27:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7fa7d77fe640 1 -- 172.21.15.105:0/1906123947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8044020 con 0x7fa7f0162480 2024-03-08T08:27:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fe60271e640 1 Processor -- start 2024-03-08T08:27:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7fa7002ae640 1 -- 172.21.15.105:0/3829372716 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f815dfc0 con 0x7fa6f816ca70 2024-03-08T08:27:48.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.804+0000 7fe60271e640 1 -- start start 2024-03-08T08:27:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7f7db61e5640 1 -- start start 2024-03-08T08:27:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7fa7f57b5640 1 -- 172.21.15.105:0/1906123947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f0163c00 con 0x7fa7f0162480 2024-03-08T08:27:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7fa7f57b5640 1 -- 172.21.15.105:0/1906123947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f01641c0 con 0x7fa7f0162480 2024-03-08T08:27:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7fa7d77fe640 1 -- 172.21.15.105:0/1906123947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e8035e50 con 0x7fa7f0162480 2024-03-08T08:27:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.805+0000 7fa7d77fe640 1 -- 172.21.15.105:0/1906123947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8039040 con 0x7fa7f0162480 2024-03-08T08:27:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fd0cabae640 1 Processor -- start 2024-03-08T08:27:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fd0cabae640 1 -- start start 2024-03-08T08:27:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fe60271e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5fc06ec20 0x7fe5fc14aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fe60271e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc15aad0 0x7fe5fc15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fe60271e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fe60271e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0728a0 con 0x7fe5fc15aad0 2024-03-08T08:27:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fe60271e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc15b330 con 0x7fe5fc06ec20 2024-03-08T08:27:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fe60271e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc06da00 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7f7db61e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fe72bfff640 1 -- 172.21.15.105:0/215681899 shutdown_connections 2024-03-08T08:27:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7f7db61e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db00cf360 0x7f7db009cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7f7db61e5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7db0162200 0x7f7db006ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7ffb7969d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c007400 0x7ffb6c17d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7f7db61e5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b4900 con 0x7f7db0162200 2024-03-08T08:27:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7f7db61e5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b4620 con 0x7f7db006c3a0 2024-03-08T08:27:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7f7db61e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b4be0 con 0x7f7db00cf360 2024-03-08T08:27:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7ffb7969d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c034850 0x7ffb6c04b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7f9a10a22640 1 Processor -- start 2024-03-08T08:27:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7f9a10a22640 1 -- start start 2024-03-08T08:27:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7ffb7969d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c0e7890 0x7ffb6c04b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7ffb7969d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0e7c70 con 0x7ffb6c007400 2024-03-08T08:27:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7ffb78e9c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c0e7890 0x7ffb6c04b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7ffb78e9c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c0e7890 0x7ffb6c04b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59760/0 (socket says 172.21.15.105:59760) 2024-03-08T08:27:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7ffb78e9c640 1 -- 172.21.15.105:0/2946028125 learned_addr learned my addr 172.21.15.105:0/2946028125 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c051020 con 0x7ffb6c0e7890 2024-03-08T08:27:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0e7df0 con 0x7ffb6c034850 2024-03-08T08:27:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7ffb737fe640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c034850 0x7ffb6c04b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7fa7f57b5640 1 -- 172.21.15.105:0/1906123947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d0005290 con 0x7fa7f0162480 2024-03-08T08:27:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db51e3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db51e3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59752/0 (socket says 172.21.15.105:59752) 2024-03-08T08:27:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db49e2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db00cf360 0x7f7db009cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db51e3640 1 -- 172.21.15.105:0/3008439852 learned_addr learned my addr 172.21.15.105:0/3008439852 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db59e4640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7db0162200 0x7f7db006ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.806+0000 7fe72bfff640 1 -- 172.21.15.105:0/215681899 wait complete. 2024-03-08T08:27:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7fe600cbb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7fe600cbb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc15b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:48948/0 (socket says 172.21.15.105:48948) 2024-03-08T08:27:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7fe600cbb640 1 -- 172.21.15.105:0/4008489405 learned_addr learned my addr 172.21.15.105:0/4008489405 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db51e3640 1 -- 172.21.15.105:0/3008439852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db00cf360 msgr2=0x7f7db009cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f9a10a22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c06b1c0 0x7f9a0c1623e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f9a10a22640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c0afd40 0x7f9a0c15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7fd0cabae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 0x7fd0c41412f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db51e3640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db00cf360 0x7f7db009cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db51e3640 1 -- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7db0162200 msgr2=0x7f7db006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db51e3640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7db0162200 0x7f7db006ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f7db51e3640 1 -- 172.21.15.105:0/3008439852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db009d480 con 0x7f7db006c3a0 2024-03-08T08:27:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.807+0000 7f9a10a22640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c0b38f0 0x7f9a0c1edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a10a22640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c15f720 con 0x7f9a0c06b1c0 2024-03-08T08:27:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a10a22640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0b5650 con 0x7f9a0c0b38f0 2024-03-08T08:27:48.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a10a22640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c15f8a0 con 0x7f9a0c0afd40 2024-03-08T08:27:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c06b1c0 0x7f9a0c1623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fd0cabae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c40cf4e0 0x7fd0c40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f7db51e3640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00ae8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00c7f0 tx=0x7f7dac00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fd0c894a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 0x7fd0c41412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fe600cbb640 1 -- 172.21.15.105:0/4008489405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5fc06ec20 msgr2=0x7fe5fc14aa70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T08:27:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fe600cbb640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5fc06ec20 0x7fe5fc14aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fe600cbb640 1 -- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc15aad0 msgr2=0x7fe5fc15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fe600cbb640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc15aad0 0x7fe5fc15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fe600cbb640 1 -- 172.21.15.105:0/4008489405 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc159610 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fd0c894a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 0x7fd0c41412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59546/0 (socket says 172.21.15.105:59546) 2024-03-08T08:27:48.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0affd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c0afd40 0x7f9a0c15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0affd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c0afd40 0x7f9a0c15e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:48994/0 (socket says 172.21.15.105:48994) 2024-03-08T08:27:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0affd640 1 -- 172.21.15.105:0/1791913364 learned_addr learned my addr 172.21.15.105:0/1791913364 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0bfff640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c0b38f0 0x7f9a0c1edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f7da5ffb640 1 -- 172.21.15.105:0/3008439852 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac015070 con 0x7f7db006c3a0 2024-03-08T08:27:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f7da5ffb640 1 -- 172.21.15.105:0/3008439852 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac004240 con 0x7f7db006c3a0 2024-03-08T08:27:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f7da5ffb640 1 -- 172.21.15.105:0/3008439852 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac007620 con 0x7f7db006c3a0 2024-03-08T08:27:48.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fe600cbb640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc15b9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5f402c750 tx=0x7fe5f4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fd0c894a640 1 -- 172.21.15.105:0/2054752939 learned_addr learned my addr 172.21.15.105:0/2054752939 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fa6eeffd640 1 -- 172.21.15.105:0/3829372716 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6e8012b50 con 0x7fa6f816ca70 2024-03-08T08:27:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fd0c3fff640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c40cf4e0 0x7fd0c40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fe72bfff640 1 Processor -- start 2024-03-08T08:27:48.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fe72bfff640 1 -- start start 2024-03-08T08:27:48.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db009d760 con 0x7f7db006c3a0 2024-03-08T08:27:48.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fd0cabae640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4145a00 0x7fd0c4143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db00ae3b0 con 0x7f7db006c3a0 2024-03-08T08:27:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c41631e0 con 0x7fd0c40716b0 2024-03-08T08:27:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0affd640 1 -- 172.21.15.105:0/1791913364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c0b38f0 msgr2=0x7f9a0c1edec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0affd640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c0b38f0 0x7f9a0c1edec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0affd640 1 -- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c06b1c0 msgr2=0x7f9a0c1623e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0affd640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c06b1c0 0x7f9a0c1623e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0affd640 1 -- 172.21.15.105:0/1791913364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c161880 con 0x7f9a0c0afd40 2024-03-08T08:27:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.808+0000 7f9a0b7fe640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c06b1c0 0x7f9a0c1623e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:48.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fd0c914b640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4145a00 0x7fd0c4143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7f9a0affd640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c0afd40 0x7f9a0c15e0e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e9b0 tx=0x7f9a0000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fa6eeffd640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa6dc06b5d0 0x7fa6dc06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c4162f00 con 0x7fd0c4145a00 2024-03-08T08:27:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c415c9a0 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7f99f3fff640 1 -- 172.21.15.105:0/1791913364 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00017070 con 0x7f9a0c0afd40 2024-03-08T08:27:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fa7002ae640 1 -- 172.21.15.105:0/3829372716 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f8075120 con 0x7fa6f816ca70 2024-03-08T08:27:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fa6eeffd640 1 -- 172.21.15.105:0/3829372716 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6e808a5a0 con 0x7fa6f816ca70 2024-03-08T08:27:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fa6fe04a640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa6dc06b5d0 0x7fa6dc06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c1607e0 con 0x7f9a0c0afd40 2024-03-08T08:27:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c161080 con 0x7f9a0c0afd40 2024-03-08T08:27:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7f99f3fff640 1 -- 172.21.15.105:0/1791913364 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0000c9a0 con 0x7f9a0c0afd40 2024-03-08T08:27:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7f99f3fff640 1 -- 172.21.15.105:0/1791913364 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000cd50 con 0x7f9a0c0afd40 2024-03-08T08:27:48.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe72c06dc30 0x7fe72c1427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fd0c3fff640 1 -- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4145a00 msgr2=0x7fd0c4143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fd0c3fff640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4145a00 0x7fd0c4143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fd0c3fff640 1 -- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 msgr2=0x7fd0c41412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fd0c3fff640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 0x7fd0c41412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fd0c3fff640 1 -- 172.21.15.105:0/2054752939 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c40ccc50 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7ffb78e9c640 1 -- 172.21.15.105:0/2946028125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c034850 msgr2=0x7ffb6c04b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7ffb78e9c640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c034850 0x7ffb6c04b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7ffb78e9c640 1 -- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c007400 msgr2=0x7ffb6c17d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7ffb78e9c640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c007400 0x7ffb6c17d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7ffb78e9c640 1 -- 172.21.15.105:0/2946028125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c04e4b0 con 0x7ffb6c0e7890 2024-03-08T08:27:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.810+0000 7fa6fe04a640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa6dc06b5d0 0x7fa6dc06da90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0000f80 tx=0x7fa6f0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.810+0000 7fa7d77fe640 1 -- 172.21.15.105:0/1906123947 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7e8047020 con 0x7fa7f0162480 2024-03-08T08:27:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.810+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db0075120 con 0x7f7db006c3a0 2024-03-08T08:27:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.810+0000 7f7da5ffb640 1 -- 172.21.15.105:0/3008439852 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7dac0077c0 con 0x7f7db006c3a0 2024-03-08T08:27:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.810+0000 7fd0c3fff640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c40cf4e0 0x7fd0c40b3f40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00c410 tx=0x7fd0ac00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fa7d77fe640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa7dc06b520 0x7fa7dc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe5f8ff9640 1 -- 172.21.15.105:0/4008489405 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4042020 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe5f8ff9640 1 -- 172.21.15.105:0/4008489405 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f4004680 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fa7d77fe640 1 -- 172.21.15.105:0/1906123947 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7e80accb0 con 0x7fa7f0162480 2024-03-08T08:27:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fa7eeffd640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa7dc06b520 0x7fa7dc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c160bd0 con 0x7f9a0c0afd40 2024-03-08T08:27:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7f7da5ffb640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7d9406b520 0x7f7d9406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7f7da5ffb640 1 -- 172.21.15.105:0/3008439852 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dac08a120 con 0x7f7db006c3a0 2024-03-08T08:27:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7f7db49e2640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7d9406b520 0x7f7d9406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.809+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe72c1441a0 0x7fe72c1f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe72c162010 0x7fe72c1f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72bfff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c06e080 con 0x7fe72c06dc30 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72bfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0ae840 con 0x7fe72c162010 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72bfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0aed60 con 0x7fe72c1441a0 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe72c1441a0 0x7fe72c1f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe72c1441a0 0x7fe72c1f46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:49012/0 (socket says 172.21.15.105:49012) 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 -- 172.21.15.105:0/348682903 learned_addr learned my addr 172.21.15.105:0/348682903 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 -- 172.21.15.105:0/348682903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe72c162010 msgr2=0x7fe72c1f4be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe72c162010 0x7fe72c1f4be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 -- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe72c06dc30 msgr2=0x7fe72c1427a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T08:27:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe72c06dc30 0x7fe72c1427a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 -- 172.21.15.105:0/348682903 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c0cebe0 con 0x7fe72c1441a0 2024-03-08T08:27:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fa7eeffd640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa7dc06b520 0x7fa7dc06d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa7f015c030 tx=0x7fa7e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5fc14b4c0 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc158b30 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe5f8ff9640 1 -- 172.21.15.105:0/4008489405 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f402cae0 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7fe72a7fc640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe72c1441a0 0x7fe72c1f46a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b3e0 tx=0x7fe72000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.811+0000 7ffb78e9c640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c0e7890 0x7ffb6c04b730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffb74000b60 tx=0x7ffb74068790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.812+0000 7f7db49e2640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7d9406b520 0x7f7d9406d9e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7db00af160 tx=0x7f7da0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.812+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc005290 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.814+0000 7fa7d77fe640 1 -- 172.21.15.105:0/1906123947 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7e804e070 con 0x7fa7f0162480 2024-03-08T08:27:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.819+0000 7fa6eeffd640 1 -- 172.21.15.105:0/3829372716 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6e800f300 con 0x7fa6f816ca70 2024-03-08T08:27:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.819+0000 7fd0c17fa640 1 -- 172.21.15.105:0/2054752939 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0074e0 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.819+0000 7fe7137fe640 1 -- 172.21.15.105:0/348682903 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720015020 con 0x7fe72c1441a0 2024-03-08T08:27:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.819+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c0cef20 con 0x7fe72c1441a0 2024-03-08T08:27:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.819+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c40ccf30 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.819+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c40cd4f0 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.819+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c0cf760 con 0x7fe72c1441a0 2024-03-08T08:27:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.820+0000 7fd0c17fa640 1 -- 172.21.15.105:0/2054752939 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac0047e0 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.820+0000 7fe7137fe640 1 -- 172.21.15.105:0/348682903 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe720011830 con 0x7fe72c1441a0 2024-03-08T08:27:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.820+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c075120 con 0x7fe72c1441a0 2024-03-08T08:27:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.820+0000 7fe7137fe640 1 -- 172.21.15.105:0/348682903 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720011c20 con 0x7fe72c1441a0 2024-03-08T08:27:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7fd0c17fa640 1 -- 172.21.15.105:0/2054752939 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac004bd0 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a8005290 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.820+0000 7f99f3fff640 1 -- 172.21.15.105:0/1791913364 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a0001a070 con 0x7f9a0c0afd40 2024-03-08T08:27:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7f99f3fff640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7f99f3fff640 1 -- 172.21.15.105:0/1791913364 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a0008a3b0 con 0x7f9a0c0afd40 2024-03-08T08:27:48.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7f9a0b7fe640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7ffb70ff9640 1 -- 172.21.15.105:0/2946028125 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7408f070 con 0x7ffb6c0e7890 2024-03-08T08:27:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7ffb70ff9640 1 -- 172.21.15.105:0/2946028125 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb740725d0 con 0x7ffb6c0e7890 2024-03-08T08:27:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7ffb70ff9640 1 -- 172.21.15.105:0/2946028125 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74073320 con 0x7ffb6c0e7890 2024-03-08T08:27:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.821+0000 7f7da5ffb640 1 -- 172.21.15.105:0/3008439852 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7dac058fa0 con 0x7f7db006c3a0 2024-03-08T08:27:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.822+0000 7f9a0b7fe640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f99f806b4d0 0x7f99f806d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a04002230 tx=0x7f9a04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.822+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c04e790 con 0x7ffb6c0e7890 2024-03-08T08:27:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.822+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c038fb0 con 0x7ffb6c0e7890 2024-03-08T08:27:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.822+0000 7fe5f8ff9640 1 -- 172.21.15.105:0/4008489405 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5f4045020 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.822+0000 7fe5f8ff9640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5e806b5a0 0x7fe5e806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.822+0000 7fe5f8ff9640 1 -- 172.21.15.105:0/4008489405 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5f40acef0 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.822+0000 7fe5fbfff640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5e806b5a0 0x7fe5e806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.823+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c005380 con 0x7ffb6c0e7890 2024-03-08T08:27:48.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.823+0000 7fe5fbfff640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5e806b5a0 0x7fe5e806da60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009560 tx=0x7fe5ec009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.824+0000 7ffb70ff9640 1 -- 172.21.15.105:0/2946028125 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb74092070 con 0x7ffb6c0e7890 2024-03-08T08:27:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.824+0000 7ffb70ff9640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffb5806b520 0x7ffb5806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.824+0000 7ffb70ff9640 1 -- 172.21.15.105:0/2946028125 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb74104d50 con 0x7ffb6c0e7890 2024-03-08T08:27:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.824+0000 7ffb73fff640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffb5806b520 0x7ffb5806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.824+0000 7ffb73fff640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffb5806b520 0x7ffb5806d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb68001510 tx=0x7ffb680078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.826+0000 7f99f3fff640 1 -- 172.21.15.105:0/1791913364 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a00058c50 con 0x7f9a0c0afd40 2024-03-08T08:27:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.827+0000 7fe7137fe640 1 -- 172.21.15.105:0/348682903 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe720018020 con 0x7fe72c1441a0 2024-03-08T08:27:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.828+0000 7fe7137fe640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe71806b4d0 0x7fe71806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.828+0000 7fe72affd640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe71806b4d0 0x7fe71806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.828+0000 7fe72affd640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe71806b4d0 0x7fe71806d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe72c072060 tx=0x7fe71c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.834+0000 7ffb70ff9640 1 -- 172.21.15.105:0/2946028125 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb740d3660 con 0x7ffb6c0e7890 2024-03-08T08:27:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.835+0000 7fe5f8ff9640 1 -- 172.21.15.105:0/4008489405 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5f404c020 con 0x7fe5fc15aeb0 2024-03-08T08:27:48.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.835+0000 7fe7137fe640 1 -- 172.21.15.105:0/348682903 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe720056ef0 con 0x7fe72c1441a0 2024-03-08T08:27:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.837+0000 7fd0c17fa640 1 -- 172.21.15.105:0/2054752939 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0ac011070 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.837+0000 7fd0c17fa640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd0a406b4d0 0x7fd0a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.838+0000 7fd0c894a640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd0a406b4d0 0x7fd0a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.839+0000 7fd0c17fa640 1 -- 172.21.15.105:0/2054752939 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0ac08a380 con 0x7fd0c40cf4e0 2024-03-08T08:27:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.839+0000 7fd0c894a640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd0a406b4d0 0x7fd0a406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0b402bef0 tx=0x7fd0b4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.840+0000 7fe7137fe640 1 -- 172.21.15.105:0/348682903 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe72001f020 con 0x7fe72c1441a0 2024-03-08T08:27:48.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:48.844+0000 7fd0c17fa640 1 -- 172.21.15.105:0/2054752939 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0ac058b90 con 0x7fd0c40cf4e0 2024-03-08T08:27:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.210+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7db00b3ed0 con 0x7f7db006c3a0 2024-03-08T08:27:49.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.211+0000 7f7da5ffb640 1 -- 172.21.15.105:0/3008439852 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7dac058fa0 con 0x7f7db006c3a0 2024-03-08T08:27:49.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.211+0000 7fa7002ae640 1 -- 172.21.15.105:0/3829372716 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa6f806dae0 con 0x7fa6f816ca70 2024-03-08T08:27:49.213 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:49.215 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:49.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.212+0000 7fa6eeffd640 1 -- 172.21.15.105:0/3829372716 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6e8059400 con 0x7fa6f816ca70 2024-03-08T08:27:49.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7d9406b520 msgr2=0x7f7d9406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7f7db61e5640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7d9406b520 0x7f7d9406d9e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7db00af160 tx=0x7f7da0009040 comp rx=0 tx=0).stop 2024-03-08T08:27:49.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 msgr2=0x7f7db00ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7f7db61e5640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00ae8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00c7f0 tx=0x7f7dac00ccb0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 shutdown_connections 2024-03-08T08:27:49.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7f7db61e5640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7d9406b520 0x7f7d9406d9e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7fa6ecfb9640 1 -- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa6dc06b5d0 msgr2=0x7fa6dc06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7fa6ecfb9640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa6dc06b5d0 0x7fa6dc06da90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0000f80 tx=0x7fa6f0030000 comp rx=0 tx=0).stop 2024-03-08T08:27:49.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.215+0000 7f7db61e5640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7db0162200 0x7f7db006ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7f7db61e5640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7db00cf360 0x7f7db009cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7f7db61e5640 1 --2- 172.21.15.105:0/3008439852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7db006c3a0 0x7f7db00ae8d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 >> 172.21.15.105:0/3008439852 conn(0x7f7db00cc950 msgr2=0x7f7db0161410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:49.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 -- 172.21.15.105:0/3829372716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f816ca70 msgr2=0x7fa6f816d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f816ca70 0x7fa6f816d290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800d530 tx=0x7fa6e800d9f0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 -- 172.21.15.105:0/3829372716 shutdown_connections 2024-03-08T08:27:49.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa6dc06b5d0 0x7fa6dc06da90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 shutdown_connections 2024-03-08T08:27:49.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7f7db61e5640 1 -- 172.21.15.105:0/3008439852 wait complete. 2024-03-08T08:27:49.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6f816ca70 0x7fa6f816d290 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6f816aab0 0x7fa6f81590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 --2- 172.21.15.105:0/3829372716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6f80b49a0 0x7fa6f8176eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 -- 172.21.15.105:0/3829372716 >> 172.21.15.105:0/3829372716 conn(0x7fa6f806ac80 msgr2=0x7fa6f80ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:49.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.216+0000 7fa6ecfb9640 1 -- 172.21.15.105:0/3829372716 shutdown_connections 2024-03-08T08:27:49.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.217+0000 7fa6ecfb9640 1 -- 172.21.15.105:0/3829372716 wait complete. 2024-03-08T08:27:49.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.217+0000 7fa7f57b5640 1 -- 172.21.15.105:0/1906123947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa7d0005850 con 0x7fa7f0162480 2024-03-08T08:27:49.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.217+0000 7fa7d77fe640 1 -- 172.21.15.105:0/1906123947 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7e807bca0 con 0x7fa7f0162480 2024-03-08T08:27:49.222 INFO:teuthology.orchestra.run.smithi105.stdout:55834574853 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.221+0000 7fa7d57fa640 1 -- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa7dc06b520 msgr2=0x7fa7dc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.221+0000 7fa7d57fa640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa7dc06b520 0x7fa7dc06d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa7f015c030 tx=0x7fa7e0009190 comp rx=0 tx=0).stop 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.221+0000 7fa7d57fa640 1 -- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f0162480 msgr2=0x7fa7f0160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.221+0000 7fa7d57fa640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f0162480 0x7fa7f0160ae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009eb0 tx=0x7fa7e8004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.222+0000 7fa7d57fa640 1 -- 172.21.15.105:0/1906123947 shutdown_connections 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.222+0000 7fa7d57fa640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa7dc06b520 0x7fa7dc06d9e0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa7f015c030 tx=0x7fa7e0009190 comp rx=0 tx=0).stop 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.222+0000 7fa7d57fa640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f0162480 0x7fa7f0160ae0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.222+0000 7fa7d57fa640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7f00cf7b0 0x7fa7f01605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.222+0000 7fa7d57fa640 1 --2- 172.21.15.105:0/1906123947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f00b4770 0x7fa7f0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.222+0000 7fa7d57fa640 1 -- 172.21.15.105:0/1906123947 >> 172.21.15.105:0/1906123947 conn(0x7fa7f00cfd10 msgr2=0x7fa7f015d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:49.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.223+0000 7fa7d57fa640 1 -- 172.21.15.105:0/1906123947 shutdown_connections 2024-03-08T08:27:49.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.223+0000 7fa7d57fa640 1 -- 172.21.15.105:0/1906123947 wait complete. 2024-03-08T08:27:49.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-03-08T08:27:49.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-03-08T08:27:49.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.237+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9a0c15f4e0 con 0x7f9a0c0afd40 2024-03-08T08:27:49.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.237+0000 7f99f3fff640 1 -- 172.21.15.105:0/1791913364 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a0005ccf0 con 0x7f9a0c0afd40 2024-03-08T08:27:49.239 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-03-08T08:27:49.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-03-08T08:27:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f99f806b4d0 msgr2=0x7f99f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f99f806b4d0 0x7f99f806d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a04002230 tx=0x7f9a04009040 comp rx=0 tx=0).stop 2024-03-08T08:27:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c0afd40 msgr2=0x7f9a0c15e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c0afd40 0x7f9a0c15e0e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000e9b0 tx=0x7f9a0000ee70 comp rx=0 tx=0).stop 2024-03-08T08:27:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 shutdown_connections 2024-03-08T08:27:49.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a0c0b38f0 0x7f9a0c1edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a0c0afd40 0x7f9a0c15e0e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 --2- 172.21.15.105:0/1791913364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a0c06b1c0 0x7f9a0c1623e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 >> 172.21.15.105:0/1791913364 conn(0x7f9a0c06bd90 msgr2=0x7f9a0c06d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 shutdown_connections 2024-03-08T08:27:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7f9a10a22640 1 -- 172.21.15.105:0/1791913364 wait complete. 2024-03-08T08:27:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe72c0cfef0 con 0x7fe72c1441a0 2024-03-08T08:27:49.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.241+0000 7fe7137fe640 1 -- 172.21.15.105:0/348682903 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe72c0cef20 con 0x7fe72c1441a0 2024-03-08T08:27:49.245 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-03-08T08:27:49.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.243+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd0a8005850 con 0x7fd0c40cf4e0 2024-03-08T08:27:49.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fd0c17fa640 1 -- 172.21.15.105:0/2054752939 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd0ac05cc30 con 0x7fd0c40cf4e0 2024-03-08T08:27:49.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe71806b4d0 msgr2=0x7fe71806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.246 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-03-08T08:27:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe71806b4d0 0x7fe71806d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe72c072060 tx=0x7fe71c008040 comp rx=0 tx=0).stop 2024-03-08T08:27:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe72c1441a0 msgr2=0x7fe72c1f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe72c1441a0 0x7fe72c1f46a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b3e0 tx=0x7fe72000b8a0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 shutdown_connections 2024-03-08T08:27:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe71806b4d0 0x7fe71806d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe72c162010 0x7fe72c1f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe72c1441a0 0x7fe72c1f46a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 --2- 172.21.15.105:0/348682903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe72c06dc30 0x7fe72c1427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 >> 172.21.15.105:0/348682903 conn(0x7fe72c06ac80 msgr2=0x7fe72c0af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 shutdown_connections 2024-03-08T08:27:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7fe72bfff640 1 -- 172.21.15.105:0/348682903 wait complete. 2024-03-08T08:27:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.244+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ffb6c039180 con 0x7ffb6c0e7890 2024-03-08T08:27:49.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.245+0000 7ffb70ff9640 1 -- 172.21.15.105:0/2946028125 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb740d7700 con 0x7ffb6c0e7890 2024-03-08T08:27:49.248 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.247+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd0a406b4d0 msgr2=0x7fd0a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.247+0000 7fd0cabae640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd0a406b4d0 0x7fd0a406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0b402bef0 tx=0x7fd0b4030000 comp rx=0 tx=0).stop 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.247+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c40cf4e0 msgr2=0x7fd0c40b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.247+0000 7fd0cabae640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c40cf4e0 0x7fd0c40b3f40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00c410 tx=0x7fd0ac00c8d0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 shutdown_connections 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7fd0cabae640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd0a406b4d0 0x7fd0a406d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7fd0cabae640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c4145a00 0x7fd0c4143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7fd0cabae640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c40cf4e0 0x7fd0c40b3f40 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00c410 tx=0x7fd0ac00c8d0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7fd0cabae640 1 --2- 172.21.15.105:0/2054752939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c40716b0 0x7fd0c41412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 >> 172.21.15.105:0/2054752939 conn(0x7fd0c406ac80 msgr2=0x7fd0c4163a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 shutdown_connections 2024-03-08T08:27:49.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7fd0cabae640 1 -- 172.21.15.105:0/2054752939 wait complete. 2024-03-08T08:27:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffb5806b520 msgr2=0x7ffb5806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffb5806b520 0x7ffb5806d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb68001510 tx=0x7ffb680078f0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c0e7890 msgr2=0x7ffb6c04b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c0e7890 0x7ffb6c04b730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffb74000b60 tx=0x7ffb74068790 comp rx=0 tx=0).stop 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 shutdown_connections 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ffb5806b520 0x7ffb5806d9e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb6c0e7890 0x7ffb6c04b730 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb6c034850 0x7ffb6c04b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 --2- 172.21.15.105:0/2946028125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb6c007400 0x7ffb6c17d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 >> 172.21.15.105:0/2946028125 conn(0x7ffb6c055bc0 msgr2=0x7ffb6c036bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 shutdown_connections 2024-03-08T08:27:49.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.248+0000 7ffb7969d640 1 -- 172.21.15.105:0/2946028125 wait complete. 2024-03-08T08:27:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.249+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe5dc005850 con 0x7fe5fc15aeb0 2024-03-08T08:27:49.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.249+0000 7fe5f8ff9640 1 -- 172.21.15.105:0/4008489405 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5f40333a0 con 0x7fe5fc15aeb0 2024-03-08T08:27:49.253 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-03-08T08:27:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5e806b5a0 msgr2=0x7fe5e806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5e806b5a0 0x7fe5e806da60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009560 tx=0x7fe5ec009190 comp rx=0 tx=0).stop 2024-03-08T08:27:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 msgr2=0x7fe5fc15b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc15b9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5f402c750 tx=0x7fe5f4004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 shutdown_connections 2024-03-08T08:27:49.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5e806b5a0 0x7fe5e806da60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5fc15aeb0 0x7fe5fc15b9f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5fc15aad0 0x7fe5fc15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 --2- 172.21.15.105:0/4008489405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5fc06ec20 0x7fe5fc14aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 >> 172.21.15.105:0/4008489405 conn(0x7fe5fc06bd90 msgr2=0x7fe5fc15dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 shutdown_connections 2024-03-08T08:27:49.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:49.252+0000 7fe60271e640 1 -- 172.21.15.105:0/4008489405 wait complete. 2024-03-08T08:27:49.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-03-08T08:27:49.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-03-08T08:27:49.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-03-08T08:27:49.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-03-08T08:27:49.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-03-08T08:27:50.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-08T08:27:50.238 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-08T08:27:50.242 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-08T08:27:50.259 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-08T08:27:50.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-08T08:27:50.268 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-08T08:27:50.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-08T08:27:50.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.385+0000 7f1ef5cb7640 1 Processor -- start 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.385+0000 7f1ef5cb7640 1 -- start start 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.386+0000 7f1ef5cb7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ee8054430 0x7f1ee810b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.386+0000 7f1ef5cb7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee8052cf0 con 0x7f1ee8101010 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.386+0000 7f1ef5cb7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee80341c0 con 0x7f1ee8033920 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.386+0000 7f1ef5cb7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee8038350 con 0x7f1ee8054430 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.387+0000 7f1ef54b6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ee8054430 0x7f1ee810b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.387+0000 7f1ef54b6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ee8054430 0x7f1ee810b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59562/0 (socket says 172.21.15.105:59562) 2024-03-08T08:27:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.387+0000 7f1ef54b6640 1 -- 172.21.15.105:0/4137438859 learned_addr learned my addr 172.21.15.105:0/4137438859 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.388+0000 7f1ef54b6640 1 -- 172.21.15.105:0/4137438859 >> v1:172.21.15.144:6789/0 conn(0x7f1ee8033920 legacy=0x7f1ee8053d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.388+0000 7f1ef54b6640 1 -- 172.21.15.105:0/4137438859 >> v1:172.21.15.105:6789/0 conn(0x7f1ee8101010 legacy=0x7f1ee8037c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.388+0000 7f1ef54b6640 1 -- 172.21.15.105:0/4137438859 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee8052f60 con 0x7f1ee8054430 2024-03-08T08:27:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.389+0000 7f1ef54b6640 1 --2- 172.21.15.105:0/4137438859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 0x7f1ee810b9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0000b60 tx=0x7f1ef009a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdc831fecd9a65c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.389+0000 7f1eef7fe640 1 -- 172.21.15.105:0/4137438859 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00a7070 con 0x7f1ee8054430 2024-03-08T08:27:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.389+0000 7f1eef7fe640 1 -- 172.21.15.105:0/4137438859 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef00a2440 con 0x7f1ee8054430 2024-03-08T08:27:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.389+0000 7f718e455640 1 Processor -- start 2024-03-08T08:27:50.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.389+0000 7f718e455640 1 -- start start 2024-03-08T08:27:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/4137438859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 msgr2=0x7f1ee810b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f1ef5cb7640 1 --2- 172.21.15.105:0/4137438859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 0x7f1ee810b9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0000b60 tx=0x7f1ef009a140 comp rx=0 tx=0).stop 2024-03-08T08:27:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/4137438859 shutdown_connections 2024-03-08T08:27:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f1ef5cb7640 1 --2- 172.21.15.105:0/4137438859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 0x7f1ee810b9f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.391+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/4137438859 >> 172.21.15.105:0/4137438859 conn(0x7f1ee8009fb0 msgr2=0x7f1ee800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f718e455640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71800e7810 0x7f7180038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f718e455640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7180006190 0x7f7180006620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f718e455640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71800e7bf0 con 0x7f71800eb2f0 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f718e455640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7180057f10 con 0x7f71800e7810 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.390+0000 7f718e455640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800581f0 con 0x7f7180006190 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.391+0000 7f7187fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71800e7810 0x7f7180038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.391+0000 7f7187fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71800e7810 0x7f7180038270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59578/0 (socket says 172.21.15.105:59578) 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.391+0000 7f7187fff640 1 -- 172.21.15.105:0/3045661342 learned_addr learned my addr 172.21.15.105:0/3045661342 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.391+0000 7f718c9f2640 1 --2- 172.21.15.105:0/3045661342 >> v2:172.21.15.144:3300/0 conn(0x7f7180006190 0x7f7180006620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.391+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/4137438859 shutdown_connections 2024-03-08T08:27:50.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.391+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/4137438859 wait complete. 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.392+0000 7f7186ffd640 1 -- 172.21.15.105:0/3045661342 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630290506 0 0) 0x7f71800e7bf0 con 0x7f71800eb2f0 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.392+0000 7f7186ffd640 1 -- 172.21.15.105:0/3045661342 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7178003520 con 0x7f71800eb2f0 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.392+0000 7f7187fff640 1 -- 172.21.15.105:0/3045661342 >> v2:172.21.15.144:3300/0 conn(0x7f7180006190 msgr2=0x7f7180006620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.392+0000 7f7187fff640 1 --2- 172.21.15.105:0/3045661342 >> v2:172.21.15.144:3300/0 conn(0x7f7180006190 0x7f7180006620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.392+0000 7f7187fff640 1 -- 172.21.15.105:0/3045661342 >> v1:172.21.15.144:6789/0 conn(0x7f71800eb2f0 legacy=0x7f71800eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.392+0000 7f7187fff640 1 -- 172.21.15.105:0/3045661342 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800584d0 con 0x7f71800e7810 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.393+0000 7f7187fff640 1 --2- 172.21.15.105:0/3045661342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71800e7810 0x7f7180038270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f718003bf50 tx=0x7f717c02bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc166a3cf6db4e77 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.393+0000 7f7186ffd640 1 -- 172.21.15.105:0/3045661342 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c039070 con 0x7f71800e7810 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.393+0000 7f7186ffd640 1 -- 172.21.15.105:0/3045661342 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c02c7b0 con 0x7f71800e7810 2024-03-08T08:27:50.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.394+0000 7f718e455640 1 -- 172.21.15.105:0/3045661342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71800e7810 msgr2=0x7f7180038270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.394+0000 7f718e455640 1 --2- 172.21.15.105:0/3045661342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71800e7810 0x7f7180038270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f718003bf50 tx=0x7f717c02bc10 comp rx=0 tx=0).stop 2024-03-08T08:27:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.394+0000 7f1ef5cb7640 1 Processor -- start 2024-03-08T08:27:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.395+0000 7f1ef5cb7640 1 -- start start 2024-03-08T08:27:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.395+0000 7f718e455640 1 -- 172.21.15.105:0/3045661342 shutdown_connections 2024-03-08T08:27:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.395+0000 7f718e455640 1 --2- 172.21.15.105:0/3045661342 >> v2:172.21.15.144:3300/0 conn(0x7f7180006190 0x7f7180006620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.395+0000 7f718e455640 1 --2- 172.21.15.105:0/3045661342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71800e7810 0x7f7180038270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.395+0000 7f718e455640 1 -- 172.21.15.105:0/3045661342 >> 172.21.15.105:0/3045661342 conn(0x7f71800559d0 msgr2=0x7f7180055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.396+0000 7f1ef5cb7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ee8033920 0x7f1ee80fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.396+0000 7f1ef5cb7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 0x7f1ee819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.396+0000 7f1ef5cb7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ee8101010 0x7f1ee80367a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.396+0000 7f1ef4cb5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ee8033920 0x7f1ee80fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.396+0000 7f1ef4cb5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ee8033920 0x7f1ee80fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59792/0 (socket says 172.21.15.105:59792) 2024-03-08T08:27:50.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.396+0000 7f1ef4cb5640 1 -- 172.21.15.105:0/2452033061 learned_addr learned my addr 172.21.15.105:0/2452033061 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.395+0000 7f718e455640 1 -- 172.21.15.105:0/3045661342 shutdown_connections 2024-03-08T08:27:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/2452033061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8053800 con 0x7f1ee8054430 2024-03-08T08:27:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/2452033061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8053520 con 0x7f1ee8033920 2024-03-08T08:27:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/2452033061 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80502a0 con 0x7f1ee8101010 2024-03-08T08:27:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1eeffff640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 0x7f1ee819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef54b6640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ee8101010 0x7f1ee80367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef54b6640 1 -- 172.21.15.105:0/2452033061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ee8033920 msgr2=0x7f1ee80fbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef54b6640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ee8033920 0x7f1ee80fbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef54b6640 1 -- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 msgr2=0x7f1ee819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef54b6640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 0x7f1ee819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f1ef54b6640 1 -- 172.21.15.105:0/2452033061 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80fca70 con 0x7f1ee8101010 2024-03-08T08:27:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.398+0000 7f1eeffff640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 0x7f1ee819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:50.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.395+0000 7f718e455640 1 -- 172.21.15.105:0/3045661342 wait complete. 2024-03-08T08:27:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.397+0000 7f718e455640 1 Processor -- start 2024-03-08T08:27:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.398+0000 7f718e455640 1 -- start start 2024-03-08T08:27:50.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f1ef54b6640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ee8101010 0x7f1ee80367a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0050510 tx=0x7f1ef0073970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f718e455640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7180006190 0x7f718004f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f718e455640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800e7810 0x7f7180039cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f718e455640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800eb2f0 0x7f7180051ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f718e455640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800ebe80 con 0x7f7180006190 2024-03-08T08:27:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f718e455640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800ec190 con 0x7f71800eb2f0 2024-03-08T08:27:50.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f718e455640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7180058cd0 con 0x7f71800e7810 2024-03-08T08:27:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f7187fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7180006190 0x7f718004f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f7187fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7180006190 0x7f718004f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59592/0 (socket says 172.21.15.105:59592) 2024-03-08T08:27:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f7187fff640 1 -- 172.21.15.105:0/2903508102 learned_addr learned my addr 172.21.15.105:0/2903508102 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.399+0000 7f71877fe640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800e7810 0x7f7180039cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f1eed7fa640 1 -- 172.21.15.105:0/2452033061 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00a7030 con 0x7f1ee8101010 2024-03-08T08:27:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/2452033061 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee80fc1d0 con 0x7f1ee8101010 2024-03-08T08:27:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/2452033061 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee80fc790 con 0x7f1ee8101010 2024-03-08T08:27:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f1eed7fa640 1 -- 172.21.15.105:0/2452033061 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef00a9070 con 0x7f1ee8101010 2024-03-08T08:27:50.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.401+0000 7f1eed7fa640 1 -- 172.21.15.105:0/2452033061 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00a1c50 con 0x7f1ee8101010 2024-03-08T08:27:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f7187fff640 1 -- 172.21.15.105:0/2903508102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800e7810 msgr2=0x7f7180039cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f7187fff640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800e7810 0x7f7180039cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f7187fff640 1 -- 172.21.15.105:0/2903508102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800eb2f0 msgr2=0x7f7180051ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T08:27:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f7187fff640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800eb2f0 0x7f7180051ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.400+0000 7f7187fff640 1 -- 172.21.15.105:0/2903508102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800d96a0 con 0x7f7180006190 2024-03-08T08:27:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.402+0000 7f7187fff640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7180006190 0x7f718004f0f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f717c02c220 tx=0x7f717c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.402+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/2452033061 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed0005290 con 0x7f1ee8101010 2024-03-08T08:27:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.402+0000 7f7184ff9640 1 -- 172.21.15.105:0/2903508102 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c039050 con 0x7f7180006190 2024-03-08T08:27:50.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.402+0000 7f7184ff9640 1 -- 172.21.15.105:0/2903508102 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c03a040 con 0x7f7180006190 2024-03-08T08:27:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.402+0000 7f7184ff9640 1 -- 172.21.15.105:0/2903508102 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c003d00 con 0x7f7180006190 2024-03-08T08:27:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.402+0000 7f718e455640 1 -- 172.21.15.105:0/2903508102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71800e9b50 con 0x7f7180006190 2024-03-08T08:27:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.402+0000 7f718e455640 1 -- 172.21.15.105:0/2903508102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71800ea0f0 con 0x7f7180006190 2024-03-08T08:27:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.405+0000 7f1eed7fa640 1 -- 172.21.15.105:0/2452033061 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ef00b4060 con 0x7f1ee8101010 2024-03-08T08:27:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.405+0000 7f1eed7fa640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1ed406b5a0 0x7f1ed406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.406+0000 7f1eed7fa640 1 -- 172.21.15.105:0/2452033061 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ef0115e40 con 0x7f1ee8101010 2024-03-08T08:27:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.406+0000 7f7184ff9640 1 -- 172.21.15.105:0/2903508102 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f717c040450 con 0x7f7180006190 2024-03-08T08:27:50.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.406+0000 7f1ef4cb5640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1ed406b5a0 0x7f1ed406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.406+0000 7f71727fc640 1 -- 172.21.15.105:0/2903508102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7160005290 con 0x7f7180006190 2024-03-08T08:27:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.407+0000 7f7184ff9640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f716c06b4d0 0x7f716c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.407+0000 7f7184ff9640 1 -- 172.21.15.105:0/2903508102 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f717c0ac8d0 con 0x7f7180006190 2024-03-08T08:27:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.408+0000 7f1ef4cb5640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1ed406b5a0 0x7f1ed406da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1edc000f60 tx=0x7f1edc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.408+0000 7f71877fe640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f716c06b4d0 0x7f716c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.409+0000 7f71877fe640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f716c06b4d0 0x7f716c06d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f718003a690 tx=0x7f7174009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.411+0000 7f1eed7fa640 1 -- 172.21.15.105:0/2452033061 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ef00e4ca0 con 0x7f1ee8101010 2024-03-08T08:27:50.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.414+0000 7f7184ff9640 1 -- 172.21.15.105:0/2903508102 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f717c07b730 con 0x7f7180006190 2024-03-08T08:27:50.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.450+0000 7f70be90c640 1 Processor -- start 2024-03-08T08:27:50.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.450+0000 7f70be90c640 1 -- start start 2024-03-08T08:27:50.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.450+0000 7f448359e640 1 Processor -- start 2024-03-08T08:27:50.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.450+0000 7f448359e640 1 -- start start 2024-03-08T08:27:50.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.451+0000 7f0e8c98b640 1 Processor -- start 2024-03-08T08:27:50.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.451+0000 7f70be90c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f70b8071990 0x7f70b8071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.451+0000 7f70be90c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70b806c9c0 0x7f70b806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.451+0000 7f70be90c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b81407b0 con 0x7f70b806e760 2024-03-08T08:27:50.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.451+0000 7f70be90c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b8072330 con 0x7f70b806c9c0 2024-03-08T08:27:50.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.451+0000 7f70be90c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b80b2b70 con 0x7f70b8071990 2024-03-08T08:27:50.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f70bcea9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70b806c9c0 0x7f70b806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f448359e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f447c0069d0 0x7f447c006e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f70bcea9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70b806c9c0 0x7f70b806ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59604/0 (socket says 172.21.15.105:59604) 2024-03-08T08:27:50.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f70bcea9640 1 -- 172.21.15.105:0/2964321891 learned_addr learned my addr 172.21.15.105:0/2964321891 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f448359e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c053710 con 0x7f447c0eacf0 2024-03-08T08:27:50.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f448359e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0539f0 con 0x7f447c005e80 2024-03-08T08:27:50.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.451+0000 7f0e8c98b640 1 -- start start 2024-03-08T08:27:50.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f0e8c98b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e881602b0 0x7f0e880cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8c98b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e880cc750 con 0x7f0e880cd510 2024-03-08T08:27:50.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8c98b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e880cca30 con 0x7f0e8815f660 2024-03-08T08:27:50.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8c98b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880ccd10 con 0x7f0e881602b0 2024-03-08T08:27:50.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8659c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e881602b0 0x7f0e880cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8659c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e881602b0 0x7f0e880cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:49036/0 (socket says 172.21.15.105:49036) 2024-03-08T08:27:50.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8659c640 1 -- 172.21.15.105:0/2918540892 learned_addr learned my addr 172.21.15.105:0/2918540892 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f448359e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c054320 con 0x7f447c0069d0 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f4481d9b640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f447c005e80 0x7f447c0062c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:40910/0 (socket says 172.21.15.105:40910) 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f4481d9b640 1 -- 172.21.15.105:0/3049525069 learned_addr learned my addr 172.21.15.105:0/3049525069 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f4482d9d640 1 --2- 172.21.15.105:0/3049525069 >> v2:172.21.15.144:3300/0 conn(0x7f447c0069d0 0x7f447c006e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728596521 0 0) 0x7f447c0539f0 con 0x7f447c005e80 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f446c003520 con 0x7f447c005e80 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071402497 0 0) 0x7f447c053710 con 0x7f447c0eacf0 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447c0539f0 con 0x7f447c0eacf0 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4240956650 0 0) 0x7f446c003520 con 0x7f447c005e80 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 >> v2:172.21.15.144:3300/0 conn(0x7f447c0069d0 msgr2=0x7f447c006e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 --2- 172.21.15.105:0/3049525069 >> v2:172.21.15.144:3300/0 conn(0x7f447c0069d0 0x7f447c006e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 >> v1:172.21.15.105:6789/0 conn(0x7f447c0eacf0 legacy=0x7f447c04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c053cd0 con 0x7f447c005e80 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f4474001670 con 0x7f447c005e80 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 >> v1:172.21.15.144:6789/0 conn(0x7f447c005e80 legacy=0x7f447c0062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c003b40 0x7f446c003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004800 0x7f446c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f446c005210 0x7f446c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c054320 con 0x7f446c003b40 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c00c160 con 0x7f446c004800 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c00c360 con 0x7f446c005210 2024-03-08T08:27:50.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f4482d9d640 1 --2- 172.21.15.105:0/3049525069 >> v2:172.21.15.144:3300/0 conn(0x7f447c0069d0 0x7f447c006e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f17fd2cd640 1 Processor -- start 2024-03-08T08:27:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f70bcea9640 1 -- 172.21.15.105:0/2964321891 >> v2:172.21.15.144:3300/0 conn(0x7f70b8071990 msgr2=0x7f70b8071df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T08:27:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f70bcea9640 1 --2- 172.21.15.105:0/2964321891 >> v2:172.21.15.144:3300/0 conn(0x7f70b8071990 0x7f70b8071df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f70bcea9640 1 -- 172.21.15.105:0/2964321891 >> v1:172.21.15.144:6789/0 conn(0x7f70b806e760 legacy=0x7f70b81400a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T08:27:50.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.452+0000 7f70bcea9640 1 -- 172.21.15.105:0/2964321891 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b806d780 con 0x7f70b806c9c0 2024-03-08T08:27:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8659c640 1 -- 172.21.15.105:0/2918540892 >> v1:172.21.15.144:6789/0 conn(0x7f0e8815f660 legacy=0x7f0e8815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8659c640 1 -- 172.21.15.105:0/2918540892 >> v1:172.21.15.105:6789/0 conn(0x7f0e880cd510 legacy=0x7f0e8815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.453+0000 7f0e8659c640 1 -- 172.21.15.105:0/2918540892 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880ccff0 con 0x7f0e881602b0 2024-03-08T08:27:50.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f70bcea9640 1 --2- 172.21.15.105:0/2964321891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b806c9c0 0x7f70b806ce50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f70b80b2800 tx=0x7f70b002bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1629e78fd99bf803 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f70b6ffd640 1 -- 172.21.15.105:0/2964321891 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b002caa0 con 0x7f70b806c9c0 2024-03-08T08:27:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f70b6ffd640 1 -- 172.21.15.105:0/2964321891 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b002cc40 con 0x7f70b806c9c0 2024-03-08T08:27:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f70b6ffd640 1 -- 172.21.15.105:0/2964321891 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b00343f0 con 0x7f70b806c9c0 2024-03-08T08:27:50.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f4481d9b640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004800 0x7f446c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f17fd2cd640 1 -- start start 2024-03-08T08:27:50.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f4482d9d640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f446c005210 0x7f446c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f448259c640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c003b40 0x7f446c003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.454+0000 7f4481d9b640 1 -- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f446c005210 msgr2=0x7f446c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f4481d9b640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f446c005210 0x7f446c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f4481d9b640 1 -- 172.21.15.105:0/3049525069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c003b40 msgr2=0x7f446c003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f4481d9b640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c003b40 0x7f446c003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f17fd2cd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f17f80b5c30 0x7f17f8166a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f17fd2cd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f815f7b0 con 0x7f17f815bea0 2024-03-08T08:27:50.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f17fd2cd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f815e710 con 0x7f17f80b4e40 2024-03-08T08:27:50.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f17fd2cd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f815e9f0 con 0x7f17f80b5c30 2024-03-08T08:27:50.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f70be90c640 1 -- 172.21.15.105:0/2964321891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b806c9c0 msgr2=0x7f70b806ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f70be90c640 1 --2- 172.21.15.105:0/2964321891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b806c9c0 0x7f70b806ce50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f70b80b2800 tx=0x7f70b002bf00 comp rx=0 tx=0).stop 2024-03-08T08:27:50.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f17f77fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f17f80b5c30 0x7f17f8166a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f17f6ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f17f815bea0 0x7f17f815df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40366/0 (socket says 172.21.15.105:40366) 2024-03-08T08:27:50.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f17f6ffd640 1 -- 172.21.15.105:0/1343883631 learned_addr learned my addr 172.21.15.105:0/1343883631 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:50.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f70be90c640 1 -- 172.21.15.105:0/2964321891 shutdown_connections 2024-03-08T08:27:50.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f70be90c640 1 --2- 172.21.15.105:0/2964321891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b806c9c0 0x7f70b806ce50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f70be90c640 1 --2- 172.21.15.105:0/2964321891 >> v2:172.21.15.144:3300/0 conn(0x7f70b8071990 0x7f70b8071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f70be90c640 1 -- 172.21.15.105:0/2964321891 >> 172.21.15.105:0/2964321891 conn(0x7f70b80b1740 msgr2=0x7f70b80b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f70be90c640 1 -- 172.21.15.105:0/2964321891 shutdown_connections 2024-03-08T08:27:50.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f4481d9b640 1 -- 172.21.15.105:0/3049525069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c053cd0 con 0x7f446c004800 2024-03-08T08:27:50.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f448259c640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c003b40 0x7f446c003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:50.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f4482d9d640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f446c005210 0x7f446c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:50.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f4482d9d640 1 -- 172.21.15.105:0/3049525069 reap_dead start 2024-03-08T08:27:50.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f0e8659c640 1 --2- 172.21.15.105:0/2918540892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e881602b0 0x7f0e880cea40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009e80 tx=0x7f0e8002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc9b2599e91a81c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f17f77fe640 1 -- 172.21.15.105:0/1343883631 >> v1:172.21.15.144:6789/0 conn(0x7f17f80b4e40 legacy=0x7f17f80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f17f77fe640 1 -- 172.21.15.105:0/1343883631 >> v1:172.21.15.105:6789/0 conn(0x7f17f815bea0 legacy=0x7f17f815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f4481d9b640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004800 0x7f446c004c80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f447400c240 tx=0x7f446c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ade166e09ad7511 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f17f77fe640 1 -- 172.21.15.105:0/1343883631 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f815ecd0 con 0x7f17f80b5c30 2024-03-08T08:27:50.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.455+0000 7f70be90c640 1 -- 172.21.15.105:0/2964321891 wait complete. 2024-03-08T08:27:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f0e85d9b640 1 -- 172.21.15.105:0/2918540892 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8002cb00 con 0x7f0e881602b0 2024-03-08T08:27:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474007d30 con 0x7f446c004800 2024-03-08T08:27:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474030070 con 0x7f446c004800 2024-03-08T08:27:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f448159a640 1 -- 172.21.15.105:0/3049525069 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474005d50 con 0x7f446c004800 2024-03-08T08:27:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.456+0000 7f0e85d9b640 1 -- 172.21.15.105:0/2918540892 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8002cca0 con 0x7f0e881602b0 2024-03-08T08:27:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f0e85d9b640 1 -- 172.21.15.105:0/2918540892 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80034610 con 0x7f0e881602b0 2024-03-08T08:27:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f17f77fe640 1 --2- 172.21.15.105:0/1343883631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 0x7f17f8166a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009720 tx=0x7f17ec02b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ae7b22af75a811 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f4d1320f640 1 Processor -- start 2024-03-08T08:27:50.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f70be90c640 1 Processor -- start 2024-03-08T08:27:50.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f17f5ffb640 1 -- 172.21.15.105:0/1343883631 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec037070 con 0x7f17f80b5c30 2024-03-08T08:27:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f17f5ffb640 1 -- 172.21.15.105:0/1343883631 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec030710 con 0x7f17f80b5c30 2024-03-08T08:27:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f0e8c98b640 1 -- 172.21.15.105:0/2918540892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e881602b0 msgr2=0x7f0e880cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f0e8c98b640 1 --2- 172.21.15.105:0/2918540892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e881602b0 0x7f0e880cea40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009e80 tx=0x7f0e8002bf60 comp rx=0 tx=0).stop 2024-03-08T08:27:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f0e8c98b640 1 -- 172.21.15.105:0/2918540892 shutdown_connections 2024-03-08T08:27:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f0e8c98b640 1 --2- 172.21.15.105:0/2918540892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e881602b0 0x7f0e880cea40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f0e8c98b640 1 -- 172.21.15.105:0/2918540892 >> 172.21.15.105:0/2918540892 conn(0x7f0e880b3460 msgr2=0x7f0e880b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f0e8c98b640 1 -- 172.21.15.105:0/2918540892 shutdown_connections 2024-03-08T08:27:50.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f0e8c98b640 1 -- 172.21.15.105:0/2918540892 wait complete. 2024-03-08T08:27:50.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f17f5ffb640 1 -- 172.21.15.105:0/1343883631 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec037070 con 0x7f17f80b5c30 2024-03-08T08:27:50.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f70be90c640 1 -- start start 2024-03-08T08:27:50.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f448359e640 1 -- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004800 msgr2=0x7f446c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f448359e640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004800 0x7f446c004c80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f447400c240 tx=0x7f446c005830 comp rx=0 tx=0).stop 2024-03-08T08:27:50.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f448359e640 1 -- 172.21.15.105:0/3049525069 shutdown_connections 2024-03-08T08:27:50.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.457+0000 7f4d1320f640 1 -- start start 2024-03-08T08:27:50.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f448359e640 1 --2- 172.21.15.105:0/3049525069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004800 0x7f446c004c80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f448359e640 1 -- 172.21.15.105:0/3049525069 >> 172.21.15.105:0/3049525069 conn(0x7f447c009fa0 msgr2=0x7f447c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f4d1320f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d0c0d29e0 0x7f4d0c06bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f4d1320f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d0c15e230 0x7f4d0c06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f7a988cc640 1 Processor -- start 2024-03-08T08:27:50.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f7a988cc640 1 -- start start 2024-03-08T08:27:50.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f4d1320f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0c0d0aa0 con 0x7f4d0c15de50 2024-03-08T08:27:50.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f4d1320f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0c0d0d80 con 0x7f4d0c0d29e0 2024-03-08T08:27:50.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f4d1320f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0c0d1060 con 0x7f4d0c15e230 2024-03-08T08:27:50.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f4d10fab640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4d0c15de50 0x7f4d0c0d22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40376/0 (socket says 172.21.15.105:40376) 2024-03-08T08:27:50.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f4d10fab640 1 -- 172.21.15.105:0/4224236954 learned_addr learned my addr 172.21.15.105:0/4224236954 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:50.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f4d0bfff640 1 --2- 172.21.15.105:0/4224236954 >> v2:172.21.15.105:3300/0 conn(0x7f4d0c0d29e0 0x7f4d0c06bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.458+0000 7f448359e640 1 -- 172.21.15.105:0/3049525069 shutdown_connections 2024-03-08T08:27:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f4d0b7fe640 1 -- 172.21.15.105:0/4224236954 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299713118 0 0) 0x7f4d0c0d0aa0 con 0x7f4d0c15de50 2024-03-08T08:27:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f4d0b7fe640 1 -- 172.21.15.105:0/4224236954 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf8003520 con 0x7f4d0c15de50 2024-03-08T08:27:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f4d0bfff640 1 -- 172.21.15.105:0/4224236954 >> v2:172.21.15.144:3300/0 conn(0x7f4d0c15e230 msgr2=0x7f4d0c06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f4d0b7fe640 1 -- 172.21.15.105:0/4224236954 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1072637940 0 0) 0x7f4cf8003520 con 0x7f4d0c15de50 2024-03-08T08:27:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f4d0bfff640 1 --2- 172.21.15.105:0/4224236954 >> v2:172.21.15.144:3300/0 conn(0x7f4d0c15e230 0x7f4d0c06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f4d0bfff640 1 -- 172.21.15.105:0/4224236954 >> v1:172.21.15.105:6789/0 conn(0x7f4d0c15de50 legacy=0x7f4d0c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f4d0bfff640 1 -- 172.21.15.105:0/4224236954 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c06b560 con 0x7f4d0c0d29e0 2024-03-08T08:27:50.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f7a988cc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a8c0e12b0 0x7f7a8c0534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1343883631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 msgr2=0x7f17f8166a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f17fd2cd640 1 --2- 172.21.15.105:0/1343883631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 0x7f17f8166a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009720 tx=0x7f17ec02b730 comp rx=0 tx=0).stop 2024-03-08T08:27:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f0e8c98b640 1 Processor -- start 2024-03-08T08:27:50.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70be90c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b806c9c0 0x7f70b815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1343883631 shutdown_connections 2024-03-08T08:27:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f17fd2cd640 1 --2- 172.21.15.105:0/1343883631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 0x7f17f8166a50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f0e8c98b640 1 -- start start 2024-03-08T08:27:50.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1343883631 >> 172.21.15.105:0/1343883631 conn(0x7f17f809aa00 msgr2=0x7f17f809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f448359e640 1 -- 172.21.15.105:0/3049525069 wait complete. 2024-03-08T08:27:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70be90c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70b806e760 0x7f70b80aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f7a988cc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c054b50 0x7f7a8c053a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f7a988cc640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c009bd0 con 0x7f7a8c0e4d30 2024-03-08T08:27:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f7a988cc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8c054fe0 con 0x7f7a8c0e12b0 2024-03-08T08:27:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.459+0000 7f7a988cc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0552c0 con 0x7f7a8c054b50 2024-03-08T08:27:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a93fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7a8c0e4d30 0x7f7a8c0e1690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:40948/0 (socket says 172.21.15.105:40948) 2024-03-08T08:27:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a93fff640 1 -- 172.21.15.105:0/445979968 learned_addr learned my addr 172.21.15.105:0/445979968 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b806c9c0 0x7f70b815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b806c9c0 0x7f70b815cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59874/0 (socket says 172.21.15.105:59874) 2024-03-08T08:27:50.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70b7fff640 1 -- 172.21.15.105:0/2016737874 learned_addr learned my addr 172.21.15.105:0/2016737874 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70be90c640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b8071990 0x7f70b8133720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b806d390 con 0x7f70b8071990 2024-03-08T08:27:50.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a92ffd640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c054b50 0x7f7a8c053a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857038867 0 0) 0x7f7a8c009bd0 con 0x7f7a8c0e4d30 2024-03-08T08:27:50.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a84003520 con 0x7f7a8c0e4d30 2024-03-08T08:27:50.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a937fe640 1 --2- 172.21.15.105:0/445979968 >> v2:172.21.15.105:3300/0 conn(0x7f7a8c0e12b0 0x7f7a8c0534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70bcea9640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b8071990 0x7f70b8133720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4257427636 0 0) 0x7f7a84003520 con 0x7f7a8c0e4d30 2024-03-08T08:27:50.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c054b50 msgr2=0x7f7a8c053a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b806dd40 con 0x7f70b806c9c0 2024-03-08T08:27:50.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f448359e640 1 Processor -- start 2024-03-08T08:27:50.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b806d510 con 0x7f70b806e760 2024-03-08T08:27:50.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f448359e640 1 -- start start 2024-03-08T08:27:50.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c054b50 0x7f7a8c053a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 >> v2:172.21.15.105:3300/0 conn(0x7f7a8c0e12b0 msgr2=0x7f7a8c0534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 --2- 172.21.15.105:0/445979968 >> v2:172.21.15.105:3300/0 conn(0x7f7a8c0e12b0 0x7f7a8c0534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c0555a0 con 0x7f7a8c0e4d30 2024-03-08T08:27:50.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f70b77fe640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70b806e760 0x7f70b80aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f0e8c98b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e880cd510 0x7f0e880c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f7a9404f4e0 con 0x7f7a8c0e4d30 2024-03-08T08:27:50.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 >> v1:172.21.15.144:6789/0 conn(0x7f7a8c0e4d30 legacy=0x7f7a8c0e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a840039b0 0x7f7a84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a84004800 0x7f7a84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.460+0000 7f7a927fc640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a840051c0 0x7f7a8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c054fe0 con 0x7f7a840051c0 2024-03-08T08:27:50.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0552c0 con 0x7f7a840039b0 2024-03-08T08:27:50.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8400c070 con 0x7f7a84004800 2024-03-08T08:27:50.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a93fff640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a840051c0 0x7f7a8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a937fe640 1 --2- 172.21.15.105:0/445979968 >> v2:172.21.15.105:3300/0 conn(0x7f7a8c0e12b0 0x7f7a8c0534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:50.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a92ffd640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c054b50 0x7f7a8c053a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:50.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a937fe640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a840039b0 0x7f7a84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1343883631 shutdown_connections 2024-03-08T08:27:50.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a92ffd640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a84004800 0x7f7a84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f70b7fff640 1 -- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70b806e760 msgr2=0x7f70b80aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f70b7fff640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70b806e760 0x7f70b80aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f70b7fff640 1 -- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b8071990 msgr2=0x7f70b8133720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f70b7fff640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b8071990 0x7f70b8133720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f70b7fff640 1 -- 172.21.15.105:0/2016737874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b81344a0 con 0x7f70b806c9c0 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f448359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c004800 0x7f447c04e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f448359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c04dab0 0x7f447c046ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f448359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c0453c0 0x7f447c047010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f448359e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c054930 con 0x7f446c004800 2024-03-08T08:27:50.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f448359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0547b0 con 0x7f447c04dab0 2024-03-08T08:27:50.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f448359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c04f540 con 0x7f447c0453c0 2024-03-08T08:27:50.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a92ffd640 1 -- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a840039b0 msgr2=0x7f7a84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a92ffd640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a840039b0 0x7f7a84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a92ffd640 1 -- 172.21.15.105:0/445979968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a840051c0 msgr2=0x7f7a8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:50.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4482d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c0453c0 0x7f447c047010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4482d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c0453c0 0x7f447c047010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:49080/0 (socket says 172.21.15.105:49080) 2024-03-08T08:27:50.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4482d9d640 1 -- 172.21.15.105:0/2751258656 learned_addr learned my addr 172.21.15.105:0/2751258656 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a92ffd640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a840051c0 0x7f7a8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f7a92ffd640 1 -- 172.21.15.105:0/445979968 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84003520 con 0x7f7a84004800 2024-03-08T08:27:50.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4481d9b640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c04dab0 0x7f447c046ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e8c98b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8815f660 0x7f0e88158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f70b77fe640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70b806e760 0x7f70b80aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:50.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f7a937fe640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a840039b0 0x7f7a84003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:50.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e8c98b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e880b4490 0x7f0e8815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f70b7fff640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b806c9c0 0x7f70b815cb50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70a800b450 tx=0x7f70a800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e8c98b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880cf100 con 0x7f0e880cd510 2024-03-08T08:27:50.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.461+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1343883631 wait complete. 2024-03-08T08:27:50.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e86d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e880cd510 0x7f0e880c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e8c98b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880cef80 con 0x7f0e8815f660 2024-03-08T08:27:50.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e86d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e880cd510 0x7f0e880c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59660/0 (socket says 172.21.15.105:59660) 2024-03-08T08:27:50.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e86d9d640 1 -- 172.21.15.105:0/4282505659 learned_addr learned my addr 172.21.15.105:0/4282505659 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e8c98b640 1 -- 172.21.15.105:0/4282505659 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8815c6d0 con 0x7f0e880b4490 2024-03-08T08:27:50.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e8659c640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8815f660 0x7f0e88158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4482d9d640 1 -- 172.21.15.105:0/2751258656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c04dab0 msgr2=0x7f447c046ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f448259c640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c004800 0x7f447c04e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4482d9d640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c04dab0 0x7f447c046ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4482d9d640 1 -- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c004800 msgr2=0x7f447c04e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4482d9d640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c004800 0x7f447c04e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f0e8759e640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e880b4490 0x7f0e8815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f70b4ff9640 1 -- 172.21.15.105:0/2016737874 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8015020 con 0x7f70b806c9c0 2024-03-08T08:27:50.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4482d9d640 1 -- 172.21.15.105:0/2751258656 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c04c730 con 0x7f447c0453c0 2024-03-08T08:27:50.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f448259c640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c004800 0x7f447c04e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:50.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f4d0bfff640 1 --2- 172.21.15.105:0/4224236954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c0d29e0 0x7f4d0c06bd40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c06d6c0 tx=0x7f4cf402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120cd66bc96f9363 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f0e8659c640 1 -- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e880b4490 msgr2=0x7f0e8815ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f0e8659c640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e880b4490 0x7f0e8815ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f0e8659c640 1 -- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e880cd510 msgr2=0x7f0e880c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f0e8659c640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e880cd510 0x7f0e880c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f0e8659c640 1 -- 172.21.15.105:0/4282505659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880c76d0 con 0x7f0e8815f660 2024-03-08T08:27:50.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f4482d9d640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c0453c0 0x7f447c047010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4484069220 tx=0x7f4484069dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f4472ffd640 1 -- 172.21.15.105:0/2751258656 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484099070 con 0x7f447c0453c0 2024-03-08T08:27:50.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f4d0b7fe640 1 -- 172.21.15.105:0/4224236954 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4038070 con 0x7f4d0c0d29e0 2024-03-08T08:27:50.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f4d0b7fe640 1 -- 172.21.15.105:0/4224236954 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf4033440 con 0x7f4d0c0d29e0 2024-03-08T08:27:50.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c04ca10 con 0x7f447c0453c0 2024-03-08T08:27:50.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f0e8659c640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8815f660 0x7f0e88158b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e80032000 tx=0x7f0e80004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f7a92ffd640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a84004800 0x7f7a84004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a88006fd0 tx=0x7f7a84005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95994a7b2f6f1f32 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f4472ffd640 1 -- 172.21.15.105:0/2751258656 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484072890 con 0x7f447c0453c0 2024-03-08T08:27:50.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f4472ffd640 1 -- 172.21.15.105:0/2751258656 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448404feb0 con 0x7f447c0453c0 2024-03-08T08:27:50.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c04d4f0 con 0x7f447c0453c0 2024-03-08T08:27:50.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f17fd2cd640 1 Processor -- start 2024-03-08T08:27:50.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f17fd2cd640 1 -- start start 2024-03-08T08:27:50.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.462+0000 7f70b4ff9640 1 -- 172.21.15.105:0/2016737874 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a8011890 con 0x7f70b806c9c0 2024-03-08T08:27:50.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b8134780 con 0x7f70b806c9c0 2024-03-08T08:27:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b81577a0 con 0x7f70b806c9c0 2024-03-08T08:27:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f70b4ff9640 1 -- 172.21.15.105:0/2016737874 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8015020 con 0x7f70b806c9c0 2024-03-08T08:27:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8802f050 con 0x7f7a84004800 2024-03-08T08:27:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88033630 con 0x7f7a84004800 2024-03-08T08:27:50.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.463+0000 7f4d0b7fe640 1 -- 172.21.15.105:0/4224236954 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d040019f0 con 0x7f4d0c0d29e0 2024-03-08T08:27:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f7a927fc640 1 -- 172.21.15.105:0/445979968 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8800f040 con 0x7f7a84004800 2024-03-08T08:27:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b80cc950 con 0x7f70b806c9c0 2024-03-08T08:27:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f0e6f7fe640 1 -- 172.21.15.105:0/4282505659 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80043020 con 0x7f0e8815f660 2024-03-08T08:27:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f0e8c98b640 1 -- 172.21.15.105:0/4282505659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e880c79b0 con 0x7f0e8815f660 2024-03-08T08:27:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f0e8c98b640 1 -- 172.21.15.105:0/4282505659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8815d250 con 0x7f0e8815f660 2024-03-08T08:27:50.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4468005290 con 0x7f447c0453c0 2024-03-08T08:27:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17fd2cd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17f80b4e40 0x7f17f814bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.464+0000 7f4d1320f640 1 -- 172.21.15.105:0/4224236954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c0d29e0 msgr2=0x7f4d0c06bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f4d1320f640 1 --2- 172.21.15.105:0/4224236954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c0d29e0 0x7f4d0c06bd40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c06d6c0 tx=0x7f4cf402b7a0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f4d1320f640 1 -- 172.21.15.105:0/4224236954 shutdown_connections 2024-03-08T08:27:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f4d1320f640 1 --2- 172.21.15.105:0/4224236954 >> v2:172.21.15.144:3300/0 conn(0x7f4d0c15e230 0x7f4d0c06c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f4d1320f640 1 --2- 172.21.15.105:0/4224236954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c0d29e0 0x7f4d0c06bd40 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c06d6c0 tx=0x7f4cf402b7a0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f4d1320f640 1 -- 172.21.15.105:0/4224236954 >> 172.21.15.105:0/4224236954 conn(0x7f4d0c09d000 msgr2=0x7f4d0c0d3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17fd2cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 0x7f17f806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17fd2cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17f815bea0 0x7f17f806d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17fd2cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f809cb00 con 0x7f17f815bea0 2024-03-08T08:27:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17fd2cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f815f290 con 0x7f17f80b5c30 2024-03-08T08:27:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17fd2cd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f815d290 con 0x7f17f80b4e40 2024-03-08T08:27:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17f6ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17f80b4e40 0x7f17f814bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17f6ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17f80b4e40 0x7f17f814bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:49104/0 (socket says 172.21.15.105:49104) 2024-03-08T08:27:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17f6ffd640 1 -- 172.21.15.105:0/1562445066 learned_addr learned my addr 172.21.15.105:0/1562445066 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17f67fc640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 0x7f17f806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.465+0000 7f17f77fe640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17f815bea0 0x7f17f806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f7a988cc640 1 -- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a84004800 msgr2=0x7f7a84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17f6ffd640 1 -- 172.21.15.105:0/1562445066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 msgr2=0x7f17f806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17f6ffd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 0x7f17f806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17f6ffd640 1 -- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17f815bea0 msgr2=0x7f17f806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17f6ffd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17f815bea0 0x7f17f806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17f6ffd640 1 -- 172.21.15.105:0/1562445066 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f814c010 con 0x7f17f80b4e40 2024-03-08T08:27:50.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f7a988cc640 1 --2- 172.21.15.105:0/445979968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a84004800 0x7f7a84004c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a88006fd0 tx=0x7f7a84005830 comp rx=0 tx=0).stop 2024-03-08T08:27:50.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f4472ffd640 1 -- 172.21.15.105:0/2751258656 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4484072c60 con 0x7f447c0453c0 2024-03-08T08:27:50.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f70b4ff9640 1 -- 172.21.15.105:0/2016737874 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70a8018020 con 0x7f70b806c9c0 2024-03-08T08:27:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17f6ffd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17f80b4e40 0x7f17f814bad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c390 tx=0x7f17e800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17df7fe640 1 -- 172.21.15.105:0/1562445066 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8017070 con 0x7f17f80b4e40 2024-03-08T08:27:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f4472ffd640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f446406b4d0 0x7f446406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f0e6f7fe640 1 -- 172.21.15.105:0/4282505659 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8002ce50 con 0x7f0e8815f660 2024-03-08T08:27:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17df7fe640 1 -- 172.21.15.105:0/1562445066 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e8005ce0 con 0x7f17f80b4e40 2024-03-08T08:27:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f70b4ff9640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f70a406b8d0 0x7f70a406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f8154df0 con 0x7f17f80b4e40 2024-03-08T08:27:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f8157330 con 0x7f17f80b4e40 2024-03-08T08:27:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.466+0000 7f70b4ff9640 1 -- 172.21.15.105:0/2016737874 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70a8011690 con 0x7f70b806c9c0 2024-03-08T08:27:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f17df7fe640 1 -- 172.21.15.105:0/1562445066 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e80072f0 con 0x7f17f80b4e40 2024-03-08T08:27:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f70b77fe640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f70a406b8d0 0x7f70a406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f4472ffd640 1 -- 172.21.15.105:0/2751258656 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44840d1560 con 0x7f447c0453c0 2024-03-08T08:27:50.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f448259c640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f446406b4d0 0x7f446406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f0e6f7fe640 1 -- 172.21.15.105:0/4282505659 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8003e650 con 0x7f0e8815f660 2024-03-08T08:27:50.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f0e6f7fe640 1 -- 172.21.15.105:0/4282505659 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e80046020 con 0x7f0e8815f660 2024-03-08T08:27:50.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f7a93fff640 1 -- 172.21.15.105:0/445979968 reap_dead start 2024-03-08T08:27:50.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f7a988cc640 1 -- 172.21.15.105:0/445979968 shutdown_connections 2024-03-08T08:27:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f70b77fe640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f70a406b8d0 0x7f70a406dd90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70b80b0aa0 tx=0x7f70ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f7a988cc640 1 -- 172.21.15.105:0/445979968 >> 172.21.15.105:0/445979968 conn(0x7f7a8c020040 msgr2=0x7f7a8c056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f448259c640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f446406b4d0 0x7f446406d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4478001340 tx=0x7f4478009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.467+0000 7f0e6f7fe640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e7406b600 0x7f0e7406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f0e86d9d640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e7406b600 0x7f0e7406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f0e6f7fe640 1 -- 172.21.15.105:0/4282505659 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e800acdc0 con 0x7f0e8815f660 2024-03-08T08:27:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f0e6d7fa640 1 -- 172.21.15.105:0/4282505659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e88075120 con 0x7f0e8815f660 2024-03-08T08:27:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f7a988cc640 1 -- 172.21.15.105:0/445979968 shutdown_connections 2024-03-08T08:27:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f0e86d9d640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e7406b600 0x7f0e7406dac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e780017a0 tx=0x7f0e78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f4d1320f640 1 -- 172.21.15.105:0/4224236954 shutdown_connections 2024-03-08T08:27:50.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f4d1320f640 1 -- 172.21.15.105:0/4224236954 wait complete. 2024-03-08T08:27:50.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d8005290 con 0x7f17f80b4e40 2024-03-08T08:27:50.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f17df7fe640 1 -- 172.21.15.105:0/1562445066 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17e801a070 con 0x7f17f80b4e40 2024-03-08T08:27:50.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.468+0000 7f7a988cc640 1 -- 172.21.15.105:0/445979968 wait complete. 2024-03-08T08:27:50.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.469+0000 7f17df7fe640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.469+0000 7f17df7fe640 1 -- 172.21.15.105:0/1562445066 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17e8089490 con 0x7f17f80b4e40 2024-03-08T08:27:50.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.469+0000 7f17f67fc640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.470+0000 7f7a988cc640 1 Processor -- start 2024-03-08T08:27:50.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.470+0000 7f17f67fc640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f17e406b4d0 0x7f17e406d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f17f806df70 tx=0x7f17f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.470+0000 7f7a988cc640 1 -- start start 2024-03-08T08:27:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.470+0000 7f4d1320f640 1 Processor -- start 2024-03-08T08:27:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.470+0000 7f4d1320f640 1 -- start start 2024-03-08T08:27:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.471+0000 7f70b4ff9640 1 -- 172.21.15.105:0/2016737874 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70b80cc950 con 0x7f70b806c9c0 2024-03-08T08:27:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.471+0000 7f7a988cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a8c04d1e0 0x7f7a8c04f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.471+0000 7f7a988cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a8c0e40d0 0x7f7a8c0398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.471+0000 7f7a988cc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c03c410 0x7f7a8c037970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.471+0000 7f7a937fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a8c04d1e0 0x7f7a8c04f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.471+0000 7f7a937fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a8c04d1e0 0x7f7a8c04f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59926/0 (socket says 172.21.15.105:59926) 2024-03-08T08:27:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d1320f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d0c15de50 0x7f4d0c0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f7a937fe640 1 -- 172.21.15.105:0/3463295655 learned_addr learned my addr 172.21.15.105:0/3463295655 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d1320f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c15e230 0x7f4d0c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d1320f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c0c8ea0 0x7f4d0c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4472ffd640 1 -- 172.21.15.105:0/2751258656 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4484091070 con 0x7f447c0453c0 2024-03-08T08:27:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d10fab640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d0c15de50 0x7f4d0c0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d10fab640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d0c15de50 0x7f4d0c0c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:49112/0 (socket says 172.21.15.105:49112) 2024-03-08T08:27:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d10fab640 1 -- 172.21.15.105:0/3362209060 learned_addr learned my addr 172.21.15.105:0/3362209060 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d1320f640 1 -- 172.21.15.105:0/3362209060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0cfc30 con 0x7f4d0c15e230 2024-03-08T08:27:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.471+0000 7f7a92ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a8c0e40d0 0x7f7a8c0398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d0bfff640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c15e230 0x7f4d0c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d117ac640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c0c8ea0 0x7f4d0c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d1320f640 1 -- 172.21.15.105:0/3362209060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0cfab0 con 0x7f4d0c0c8ea0 2024-03-08T08:27:50.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f7a93fff640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c03c410 0x7f7a8c037970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d1320f640 1 -- 172.21.15.105:0/3362209060 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0d0530 con 0x7f4d0c15de50 2024-03-08T08:27:50.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f7a988cc640 1 -- 172.21.15.105:0/3463295655 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c052c80 con 0x7f7a8c0e40d0 2024-03-08T08:27:50.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f7a988cc640 1 -- 172.21.15.105:0/3463295655 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c052b00 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f7a988cc640 1 -- 172.21.15.105:0/3463295655 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0576a0 con 0x7f7a8c03c410 2024-03-08T08:27:50.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d0bfff640 1 -- 172.21.15.105:0/3362209060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d0c15de50 msgr2=0x7f4d0c0c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.472+0000 7f4d0bfff640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d0c15de50 0x7f4d0c0c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d0bfff640 1 -- 172.21.15.105:0/3362209060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c0c8ea0 msgr2=0x7f4d0c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d0bfff640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c0c8ea0 0x7f4d0c0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f7a937fe640 1 -- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c03c410 msgr2=0x7f7a8c037970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f7a937fe640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c03c410 0x7f7a8c037970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d0bfff640 1 -- 172.21.15.105:0/3362209060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c0c7e00 con 0x7f4d0c15e230 2024-03-08T08:27:50.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f7a937fe640 1 -- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a8c0e40d0 msgr2=0x7f7a8c0398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f7a937fe640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a8c0e40d0 0x7f7a8c0398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d117ac640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c0c8ea0 0x7f4d0c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:50.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f7a937fe640 1 -- 172.21.15.105:0/3463295655 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c0e1830 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f7a93fff640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c03c410 0x7f7a8c037970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:50.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f7a937fe640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a8c04d1e0 0x7f7a8c04f950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a80006df0 tx=0x7f7a8000d590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d0bfff640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c15e230 0x7f4d0c0ca0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c08cbb0 tx=0x7f4cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d097fa640 1 -- 172.21.15.105:0/3362209060 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4043070 con 0x7f4d0c15e230 2024-03-08T08:27:50.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d097fa640 1 -- 172.21.15.105:0/3362209060 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf403e850 con 0x7f4d0c15e230 2024-03-08T08:27:50.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d097fa640 1 -- 172.21.15.105:0/3362209060 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4038040 con 0x7f4d0c15e230 2024-03-08T08:27:50.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d1320f640 1 -- 172.21.15.105:0/3362209060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c0c80e0 con 0x7f4d0c15e230 2024-03-08T08:27:50.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.473+0000 7f4d1320f640 1 -- 172.21.15.105:0/3362209060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c0c86a0 con 0x7f4d0c15e230 2024-03-08T08:27:50.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.475+0000 7f4d02ffd640 1 -- 172.21.15.105:0/3362209060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c075120 con 0x7f4d0c15e230 2024-03-08T08:27:50.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.475+0000 7f7a7ffff640 1 -- 172.21.15.105:0/3463295655 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000de40 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.475+0000 7f7a7ffff640 1 -- 172.21.15.105:0/3463295655 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a80013440 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.475+0000 7f7a7ffff640 1 -- 172.21.15.105:0/3463295655 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800137f0 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.475+0000 7f7a988cc640 1 -- 172.21.15.105:0/3463295655 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c04f150 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.475+0000 7f0e6f7fe640 1 -- 172.21.15.105:0/4282505659 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e8004d030 con 0x7f0e8815f660 2024-03-08T08:27:50.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.476+0000 7f7a988cc640 1 -- 172.21.15.105:0/3463295655 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c050b80 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.477+0000 7f17df7fe640 1 -- 172.21.15.105:0/1562445066 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17e8057cb0 con 0x7f17f80b4e40 2024-03-08T08:27:50.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.476+0000 7f4d097fa640 1 -- 172.21.15.105:0/3362209060 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cf4046070 con 0x7f4d0c15e230 2024-03-08T08:27:50.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.476+0000 7f4d097fa640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4cec06b6d0 0x7f4cec06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.476+0000 7f4d097fa640 1 -- 172.21.15.105:0/3362209060 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cf403ea00 con 0x7f4d0c15e230 2024-03-08T08:27:50.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.478+0000 7f7a7dffb640 1 -- 172.21.15.105:0/3463295655 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8c0057e0 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.477+0000 7f4d10fab640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4cec06b6d0 0x7f4cec06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.480+0000 7f4d10fab640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4cec06b6d0 0x7f4cec06db90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d04002230 tx=0x7f4d04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.480+0000 7f7a7ffff640 1 -- 172.21.15.105:0/3463295655 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a80013990 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.480+0000 7f7a7ffff640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a7806b4d0 0x7f7a7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:50.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.480+0000 7f7a7ffff640 1 -- 172.21.15.105:0/3463295655 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a8008adb0 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.481+0000 7f7a92ffd640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a7806b4d0 0x7f7a7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:50.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.482+0000 7f7a92ffd640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a7806b4d0 0x7f7a7806d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a88036000 tx=0x7f7a88031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:50.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.483+0000 7f4d097fa640 1 -- 172.21.15.105:0/3362209060 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4cf407a290 con 0x7f4d0c15e230 2024-03-08T08:27:50.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.485+0000 7f7a7ffff640 1 -- 172.21.15.105:0/3463295655 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a80017030 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.805+0000 7f1ef5cb7640 1 -- 172.21.15.105:0/2452033061 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1ed0005580 con 0x7f1ee8101010 2024-03-08T08:27:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.806+0000 7f1eed7fa640 1 -- 172.21.15.105:0/2452033061 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ef00e45a0 con 0x7f1ee8101010 2024-03-08T08:27:50.808 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2024-03-08T08:27:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.809+0000 7f1edaffd640 1 -- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1ed406b5a0 msgr2=0x7f1ed406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.809+0000 7f1edaffd640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1ed406b5a0 0x7f1ed406da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1edc000f60 tx=0x7f1edc009040 comp rx=0 tx=0).stop 2024-03-08T08:27:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.809+0000 7f1edaffd640 1 -- 172.21.15.105:0/2452033061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ee8101010 msgr2=0x7f1ee80367a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.809+0000 7f1edaffd640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ee8101010 0x7f1ee80367a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0050510 tx=0x7f1ef0073970 comp rx=0 tx=0).stop 2024-03-08T08:27:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.810+0000 7f1edaffd640 1 -- 172.21.15.105:0/2452033061 shutdown_connections 2024-03-08T08:27:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.810+0000 7f1edaffd640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1ed406b5a0 0x7f1ed406da60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.810+0000 7f1edaffd640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ee8101010 0x7f1ee80367a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.810+0000 7f1edaffd640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ee8054430 0x7f1ee819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.810+0000 7f1edaffd640 1 --2- 172.21.15.105:0/2452033061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ee8033920 0x7f1ee80fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.810+0000 7f1edaffd640 1 -- 172.21.15.105:0/2452033061 >> 172.21.15.105:0/2452033061 conn(0x7f1ee8009fb0 msgr2=0x7f1ee80049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.810+0000 7f1edaffd640 1 -- 172.21.15.105:0/2452033061 shutdown_connections 2024-03-08T08:27:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.810+0000 7f1edaffd640 1 -- 172.21.15.105:0/2452033061 wait complete. 2024-03-08T08:27:50.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.816+0000 7f71727fc640 1 -- 172.21.15.105:0/2903508102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7160005580 con 0x7f7180006190 2024-03-08T08:27:50.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.816+0000 7f7184ff9640 1 -- 172.21.15.105:0/2903508102 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f717c07b030 con 0x7f7180006190 2024-03-08T08:27:50.818 INFO:teuthology.orchestra.run.smithi105.stdout:55834574856 2024-03-08T08:27:50.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.819+0000 7f718e455640 1 -- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f716c06b4d0 msgr2=0x7f716c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f716c06b4d0 0x7f716c06d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f718003a690 tx=0x7f7174009040 comp rx=0 tx=0).stop 2024-03-08T08:27:50.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 -- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7180006190 msgr2=0x7f718004f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7180006190 0x7f718004f0f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f717c02c220 tx=0x7f717c0046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 -- 172.21.15.105:0/2903508102 shutdown_connections 2024-03-08T08:27:50.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f716c06b4d0 0x7f716c06d990 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f718003a690 tx=0x7f7174009040 comp rx=0 tx=0).stop 2024-03-08T08:27:50.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800eb2f0 0x7f7180051ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800e7810 0x7f7180039cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 --2- 172.21.15.105:0/2903508102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7180006190 0x7f718004f0f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.820+0000 7f718e455640 1 -- 172.21.15.105:0/2903508102 >> 172.21.15.105:0/2903508102 conn(0x7f71800559d0 msgr2=0x7f718003c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.821+0000 7f718e455640 1 -- 172.21.15.105:0/2903508102 shutdown_connections 2024-03-08T08:27:50.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.823+0000 7f718e455640 1 -- 172.21.15.105:0/2903508102 wait complete. 2024-03-08T08:27:50.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-03-08T08:27:50.829 DEBUG:teuthology.parallel:result is None 2024-03-08T08:27:50.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-03-08T08:27:50.841 DEBUG:teuthology.parallel:result is None 2024-03-08T08:27:50.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.848+0000 7f0e6d7fa640 1 -- 172.21.15.105:0/4282505659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0e88067680 con 0x7f0e8815f660 2024-03-08T08:27:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.849+0000 7f0e6f7fe640 1 -- 172.21.15.105:0/4282505659 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e8007bc20 con 0x7f0e8815f660 2024-03-08T08:27:50.851 INFO:teuthology.orchestra.run.smithi105.stdout:55834574854 2024-03-08T08:27:50.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 -- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e7406b600 msgr2=0x7f0e7406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e7406b600 0x7f0e7406dac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e780017a0 tx=0x7f0e78009040 comp rx=0 tx=0).stop 2024-03-08T08:27:50.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 -- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8815f660 msgr2=0x7f0e88158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8815f660 0x7f0e88158b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e80032000 tx=0x7f0e80004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 -- 172.21.15.105:0/4282505659 shutdown_connections 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e7406b600 0x7f0e7406dac0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e880b4490 0x7f0e8815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e8815f660 0x7f0e88158b60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 --2- 172.21.15.105:0/4282505659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e880cd510 0x7f0e880c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 -- 172.21.15.105:0/4282505659 >> 172.21.15.105:0/4282505659 conn(0x7f0e880b3460 msgr2=0x7f0e880c43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 -- 172.21.15.105:0/4282505659 shutdown_connections 2024-03-08T08:27:50.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.851+0000 7f0e6d7fa640 1 -- 172.21.15.105:0/4282505659 wait complete. 2024-03-08T08:27:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.855+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4468005850 con 0x7f447c0453c0 2024-03-08T08:27:50.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.855+0000 7f4472ffd640 1 -- 172.21.15.105:0/2751258656 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44840d34c0 con 0x7f447c0453c0 2024-03-08T08:27:50.857 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f446406b4d0 msgr2=0x7f446406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f446406b4d0 0x7f446406d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4478001340 tx=0x7f4478009040 comp rx=0 tx=0).stop 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c0453c0 msgr2=0x7f447c047010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c0453c0 0x7f447c047010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4484069220 tx=0x7f4484069dd0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 shutdown_connections 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f446406b4d0 0x7f446406d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f447c0453c0 0x7f447c047010 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f447c04dab0 0x7f447c046ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 --2- 172.21.15.105:0/2751258656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f446c004800 0x7f447c04e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 >> 172.21.15.105:0/2751258656 conn(0x7f447c009fa0 msgr2=0x7f447c007030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 shutdown_connections 2024-03-08T08:27:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.858+0000 7f448359e640 1 -- 172.21.15.105:0/2751258656 wait complete. 2024-03-08T08:27:50.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-03-08T08:27:50.865 DEBUG:teuthology.parallel:result is None 2024-03-08T08:27:50.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-03-08T08:27:50.871 DEBUG:teuthology.parallel:result is None 2024-03-08T08:27:50.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.880+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f70b8075120 con 0x7f70b806c9c0 2024-03-08T08:27:50.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.881+0000 7f70b4ff9640 1 -- 172.21.15.105:0/2016737874 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70a801f020 con 0x7f70b806c9c0 2024-03-08T08:27:50.883 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f70a406b8d0 msgr2=0x7f70a406dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f70a406b8d0 0x7f70a406dd90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70b80b0aa0 tx=0x7f70ac008040 comp rx=0 tx=0).stop 2024-03-08T08:27:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b806c9c0 msgr2=0x7f70b815cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b806c9c0 0x7f70b815cb50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70a800b450 tx=0x7f70a800b910 comp rx=0 tx=0).stop 2024-03-08T08:27:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 shutdown_connections 2024-03-08T08:27:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f70a406b8d0 0x7f70a406dd90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70b8071990 0x7f70b8133720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70b806e760 0x7f70b80aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 --2- 172.21.15.105:0/2016737874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70b806c9c0 0x7f70b815cb50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 >> 172.21.15.105:0/2016737874 conn(0x7f70b80b1740 msgr2=0x7f70b80ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 shutdown_connections 2024-03-08T08:27:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.884+0000 7f70be90c640 1 -- 172.21.15.105:0/2016737874 wait complete. 2024-03-08T08:27:50.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.888+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f17d8005580 con 0x7f17f80b4e40 2024-03-08T08:27:50.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.888+0000 7f17df7fe640 1 -- 172.21.15.105:0/1562445066 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17e805bd50 con 0x7f17f80b4e40 2024-03-08T08:27:50.890 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f17e406b4d0 msgr2=0x7f17e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f17e406b4d0 0x7f17e406d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f17f806df70 tx=0x7f17f0009040 comp rx=0 tx=0).stop 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17f80b4e40 msgr2=0x7f17f814bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17f80b4e40 0x7f17f814bad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c390 tx=0x7f17e800c850 comp rx=0 tx=0).stop 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 shutdown_connections 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17f815bea0 0x7f17f806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80b5c30 0x7f17f806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 --2- 172.21.15.105:0/1562445066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17f80b4e40 0x7f17f814bad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 >> 172.21.15.105:0/1562445066 conn(0x7f17f809aa00 msgr2=0x7f17f815aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 shutdown_connections 2024-03-08T08:27:50.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.890+0000 7f17fd2cd640 1 -- 172.21.15.105:0/1562445066 wait complete. 2024-03-08T08:27:50.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-03-08T08:27:50.898 DEBUG:teuthology.parallel:result is None 2024-03-08T08:27:50.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.899+0000 7f4d02ffd640 1 -- 172.21.15.105:0/3362209060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4d0c0b9770 con 0x7f4d0c15e230 2024-03-08T08:27:50.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.900+0000 7f4d097fa640 1 -- 172.21.15.105:0/3362209060 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4cf407e330 con 0x7f4d0c15e230 2024-03-08T08:27:50.902 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:50.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-03-08T08:27:50.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.902+0000 7f7a7dffb640 1 -- 172.21.15.105:0/3463295655 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7a8c057480 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.904 DEBUG:teuthology.parallel:result is None 2024-03-08T08:27:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.902+0000 7f7a7ffff640 1 -- 172.21.15.105:0/3463295655 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a80017030 con 0x7f7a8c04d1e0 2024-03-08T08:27:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 -- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4cec06b6d0 msgr2=0x7f4cec06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.905 INFO:teuthology.orchestra.run.smithi105.stdout:55834574855 2024-03-08T08:27:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4cec06b6d0 0x7f4cec06db90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4d04002230 tx=0x7f4d04009040 comp rx=0 tx=0).stop 2024-03-08T08:27:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 -- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c15e230 msgr2=0x7f4d0c0ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c15e230 0x7f4d0c0ca0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c08cbb0 tx=0x7f4cf40046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 -- 172.21.15.105:0/3362209060 shutdown_connections 2024-03-08T08:27:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4cec06b6d0 0x7f4cec06db90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c0c8ea0 0x7f4d0c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d0c15e230 0x7f4d0c0ca0c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 --2- 172.21.15.105:0/3362209060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d0c15de50 0x7f4d0c0c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 -- 172.21.15.105:0/3362209060 >> 172.21.15.105:0/3362209060 conn(0x7f4d0c09d000 msgr2=0x7f4d0c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 -- 172.21.15.105:0/3362209060 shutdown_connections 2024-03-08T08:27:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.903+0000 7f4d02ffd640 1 -- 172.21.15.105:0/3362209060 wait complete. 2024-03-08T08:27:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 -- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a7806b4d0 msgr2=0x7f7a7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a7806b4d0 0x7f7a7806d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a88036000 tx=0x7f7a88031000 comp rx=0 tx=0).stop 2024-03-08T08:27:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 -- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a8c04d1e0 msgr2=0x7f7a8c04f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a8c04d1e0 0x7f7a8c04f950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a80006df0 tx=0x7f7a8000d590 comp rx=0 tx=0).stop 2024-03-08T08:27:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 -- 172.21.15.105:0/3463295655 shutdown_connections 2024-03-08T08:27:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a7806b4d0 0x7f7a7806d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a8c03c410 0x7f7a8c037970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a8c0e40d0 0x7f7a8c0398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 --2- 172.21.15.105:0/3463295655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a8c04d1e0 0x7f7a8c04f950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 -- 172.21.15.105:0/3463295655 >> 172.21.15.105:0/3463295655 conn(0x7f7a8c020040 msgr2=0x7f7a8c04a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 -- 172.21.15.105:0/3463295655 shutdown_connections 2024-03-08T08:27:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:50.905+0000 7f7a7dffb640 1 -- 172.21.15.105:0/3463295655 wait complete. 2024-03-08T08:27:50.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-03-08T08:27:50.916 DEBUG:teuthology.parallel:result is None 2024-03-08T08:27:50.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-03-08T08:27:50.918 DEBUG:teuthology.parallel:result is None 2024-03-08T08:27:50.918 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-08T08:27:50.918 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:27:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.056+0000 7f9f2a7a2640 1 Processor -- start 2024-03-08T08:27:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.057+0000 7f9f2a7a2640 1 -- start start 2024-03-08T08:27:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.057+0000 7f9f2a7a2640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9f2415a710 0x7f9f2415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.057+0000 7f9f2a7a2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240b41a0 0x7f9f240b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.057+0000 7f9f2a7a2640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2415f340 con 0x7f9f240afa30 2024-03-08T08:27:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.057+0000 7f9f2a7a2640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f240b4af0 con 0x7f9f2415a710 2024-03-08T08:27:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.057+0000 7f9f2a7a2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240b3930 con 0x7f9f240b41a0 2024-03-08T08:27:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f29fa1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240b41a0 0x7f9f240b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f297a0640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9f2415a710 0x7f9f2415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f28f9f640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9f240afa30 0x7f9f2406e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:40964/0 (socket says 172.21.15.105:40964) 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f28f9f640 1 -- 172.21.15.105:0/1131940786 learned_addr learned my addr 172.21.15.105:0/1131940786 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f29fa1640 1 -- 172.21.15.105:0/1131940786 >> v2:172.21.15.144:3300/0 conn(0x7f9f2415a710 msgr2=0x7f9f2415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f29fa1640 1 --2- 172.21.15.105:0/1131940786 >> v2:172.21.15.144:3300/0 conn(0x7f9f2415a710 0x7f9f2415ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f29fa1640 1 -- 172.21.15.105:0/1131940786 >> v1:172.21.15.144:6789/0 conn(0x7f9f240afa30 legacy=0x7f9f2406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f29fa1640 1 -- 172.21.15.105:0/1131940786 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f240b3c10 con 0x7f9f240b41a0 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f1bfff640 1 -- 172.21.15.105:0/1131940786 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978256638 0 0) 0x7f9f14001110 con 0x7f9f240afa30 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.058+0000 7f9f297a0640 1 --2- 172.21.15.105:0/1131940786 >> v2:172.21.15.144:3300/0 conn(0x7f9f2415a710 0x7f9f2415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:51.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.059+0000 7f9f29fa1640 1 --2- 172.21.15.105:0/1131940786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240b41a0 0x7f9f240b45b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009d90 tx=0x7f9f1c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b457956a72c97676 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.059+0000 7f9f1bfff640 1 -- 172.21.15.105:0/1131940786 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c039070 con 0x7f9f240b41a0 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.059+0000 7f9f1bfff640 1 -- 172.21.15.105:0/1131940786 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c02cc50 con 0x7f9f240b41a0 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.059+0000 7f9f1bfff640 1 -- 172.21.15.105:0/1131940786 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c034440 con 0x7f9f240b41a0 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.060+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/1131940786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240b41a0 msgr2=0x7f9f240b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.060+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/1131940786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240b41a0 0x7f9f240b45b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009d90 tx=0x7f9f1c02bea0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.060+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/1131940786 shutdown_connections 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.060+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/1131940786 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240b41a0 0x7f9f240b45b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.060+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/1131940786 >> v2:172.21.15.144:3300/0 conn(0x7f9f2415a710 0x7f9f2415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.060+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/1131940786 >> 172.21.15.105:0/1131940786 conn(0x7f9f2406bd90 msgr2=0x7f9f2406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.060+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/1131940786 shutdown_connections 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.060+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/1131940786 wait complete. 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.061+0000 7f9f2a7a2640 1 Processor -- start 2024-03-08T08:27:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.061+0000 7f9f2a7a2640 1 -- start start 2024-03-08T08:27:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.061+0000 7f9f2a7a2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240afa30 0x7f9f240ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.061+0000 7f9f2a7a2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f240b41a0 0x7f9f24148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.061+0000 7f9f2a7a2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f2415a710 0x7f9f24158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.061+0000 7f9f2a7a2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2406ebf0 con 0x7f9f2415a710 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.061+0000 7f9f2a7a2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2406dfc0 con 0x7f9f240b41a0 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.061+0000 7f9f2a7a2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2415aaf0 con 0x7f9f240afa30 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f2415a710 0x7f9f24158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f2415a710 0x7f9f24158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59710/0 (socket says 172.21.15.105:59710) 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 -- 172.21.15.105:0/3817360979 learned_addr learned my addr 172.21.15.105:0/3817360979 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f28f9f640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f240b41a0 0x7f9f24148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f297a0640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240afa30 0x7f9f240ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 -- 172.21.15.105:0/3817360979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240afa30 msgr2=0x7f9f240ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240afa30 0x7f9f240ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 -- 172.21.15.105:0/3817360979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f240b41a0 msgr2=0x7f9f24148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f240b41a0 0x7f9f24148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 -- 172.21.15.105:0/3817360979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f241487a0 con 0x7f9f2415a710 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f29fa1640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f2415a710 0x7f9f24158c40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009d90 tx=0x7f9f1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f19ffb640 1 -- 172.21.15.105:0/3817360979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c039050 con 0x7f9f2415a710 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f19ffb640 1 -- 172.21.15.105:0/3817360979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c045070 con 0x7f9f2415a710 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f19ffb640 1 -- 172.21.15.105:0/3817360979 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c009450 con 0x7f9f2415a710 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.062+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f240cd770 con 0x7f9f2415a710 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.063+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f240cdd10 con 0x7f9f2415a710 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.064+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2404efc0 con 0x7f9f2415a710 2024-03-08T08:27:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.064+0000 7f9f19ffb640 1 -- 172.21.15.105:0/3817360979 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f1c03e610 con 0x7f9f2415a710 2024-03-08T08:27:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.064+0000 7f9f19ffb640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f0806b500 0x7f9f0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.064+0000 7f9f19ffb640 1 -- 172.21.15.105:0/3817360979 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f1c0ac720 con 0x7f9f2415a710 2024-03-08T08:27:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.064+0000 7f9f297a0640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f0806b500 0x7f9f0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.065+0000 7f9f297a0640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f0806b500 0x7f9f0806d9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009a30 tx=0x7f9f20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.069+0000 7f9f19ffb640 1 -- 172.21.15.105:0/3817360979 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f1c04e020 con 0x7f9f2415a710 2024-03-08T08:27:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.239+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f241558a0 con 0x7f9f0806b500 2024-03-08T08:27:51.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.245+0000 7f9f19ffb640 1 -- 172.21.15.105:0/3817360979 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251837 (secure 0 0 0) 0x7f9f241558a0 con 0x7f9f0806b500 2024-03-08T08:27:51.250 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:51.251 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:27:51.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f0806b500 msgr2=0x7f9f0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f0806b500 0x7f9f0806d9c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009a30 tx=0x7f9f20008040 comp rx=0 tx=0).stop 2024-03-08T08:27:51.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f2415a710 msgr2=0x7f9f24158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f2415a710 0x7f9f24158c40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009d90 tx=0x7f9f1c004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 shutdown_connections 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f0806b500 0x7f9f0806d9c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f2415a710 0x7f9f24158c40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f240b41a0 0x7f9f24148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 --2- 172.21.15.105:0/3817360979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f240afa30 0x7f9f240ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 >> 172.21.15.105:0/3817360979 conn(0x7f9f2406bd90 msgr2=0x7f9f2415e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 shutdown_connections 2024-03-08T08:27:51.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.248+0000 7f9f2a7a2640 1 -- 172.21.15.105:0/3817360979 wait complete. 2024-03-08T08:27:51.261 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-03-08T08:27:51.098285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.434491"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285490+0000","last_change":"2024-03-08T08:27:39.669170+0000","last_active":"2024-03-08T08:27:42.285490+0000","last_peered":"2024-03-08T08:27:42.285490+0000","last_clean":"2024-03-08T08:27:42.285490+0000","last_became_active":"2024-03-08T08:27:39.668776+0000","last_became_peered":"2024-03-08T08:27:39.668776+0000","last_unstale":"2024-03-08T08:27:42.285490+0000","last_undegraded":"2024-03-08T08:27:42.285490+0000","last_fullsized":"2024-03-08T08:27:42.285490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:58.646243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665542+0000","last_change":"2024-03-08T08:27:39.669937+0000","last_active":"2024-03-08T08:27:40.665542+0000","last_peered":"2024-03-08T08:27:40.665542+0000","last_clean":"2024-03-08T08:27:40.665542+0000","last_became_active":"2024-03-08T08:27:39.669643+0000","last_became_peered":"2024-03-08T08:27:39.669643+0000","last_unstale":"2024-03-08T08:27:40.665542+0000","last_undegraded":"2024-03-08T08:27:40.665542+0000","last_fullsized":"2024-03-08T08:27:40.665542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229937+0000","last_change":"2024-03-08T08:27:39.669698+0000","last_active":"2024-03-08T08:27:43.229937+0000","last_peered":"2024-03-08T08:27:43.229937+0000","last_clean":"2024-03-08T08:27:43.229937+0000","last_became_active":"2024-03-08T08:27:39.669412+0000","last_became_peered":"2024-03-08T08:27:39.669412+0000","last_unstale":"2024-03-08T08:27:43.229937+0000","last_undegraded":"2024-03-08T08:27:43.229937+0000","last_fullsized":"2024-03-08T08:27:43.229937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229311+0000","last_change":"2024-03-08T08:27:39.674216+0000","last_active":"2024-03-08T08:27:43.229311+0000","last_peered":"2024-03-08T08:27:43.229311+0000","last_clean":"2024-03-08T08:27:43.229311+0000","last_became_active":"2024-03-08T08:27:39.673664+0000","last_became_peered":"2024-03-08T08:27:39.673664+0000","last_unstale":"2024-03-08T08:27:43.229311+0000","last_undegraded":"2024-03-08T08:27:43.229311+0000","last_fullsized":"2024-03-08T08:27:43.229311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285273+0000","last_change":"2024-03-08T08:27:39.671956+0000","last_active":"2024-03-08T08:27:42.285273+0000","last_peered":"2024-03-08T08:27:42.285273+0000","last_clean":"2024-03-08T08:27:42.285273+0000","last_became_active":"2024-03-08T08:27:39.671671+0000","last_became_peered":"2024-03-08T08:27:39.671671+0000","last_unstale":"2024-03-08T08:27:42.285273+0000","last_undegraded":"2024-03-08T08:27:42.285273+0000","last_fullsized":"2024-03-08T08:27:42.285273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229106+0000","last_change":"2024-03-08T08:27:39.681569+0000","last_active":"2024-03-08T08:27:43.229106+0000","last_peered":"2024-03-08T08:27:43.229106+0000","last_clean":"2024-03-08T08:27:43.229106+0000","last_became_active":"2024-03-08T08:27:39.681306+0000","last_became_peered":"2024-03-08T08:27:39.681306+0000","last_unstale":"2024-03-08T08:27:43.229106+0000","last_undegraded":"2024-03-08T08:27:43.229106+0000","last_fullsized":"2024-03-08T08:27:43.229106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285103+0000","last_change":"2024-03-08T08:27:39.669471+0000","last_active":"2024-03-08T08:27:42.285103+0000","last_peered":"2024-03-08T08:27:42.285103+0000","last_clean":"2024-03-08T08:27:42.285103+0000","last_became_active":"2024-03-08T08:27:39.669144+0000","last_became_peered":"2024-03-08T08:27:39.669144+0000","last_unstale":"2024-03-08T08:27:42.285103+0000","last_undegraded":"2024-03-08T08:27:42.285103+0000","last_fullsized":"2024-03-08T08:27:42.285103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228726+0000","last_change":"2024-03-08T08:27:39.680587+0000","last_active":"2024-03-08T08:27:43.228726+0000","last_peered":"2024-03-08T08:27:43.228726+0000","last_clean":"2024-03-08T08:27:43.228726+0000","last_became_active":"2024-03-08T08:27:39.680354+0000","last_became_peered":"2024-03-08T08:27:39.680354+0000","last_unstale":"2024-03-08T08:27:43.228726+0000","last_undegraded":"2024-03-08T08:27:43.228726+0000","last_fullsized":"2024-03-08T08:27:43.228726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285685+0000","last_change":"2024-03-08T08:27:39.682744+0000","last_active":"2024-03-08T08:27:42.285685+0000","last_peered":"2024-03-08T08:27:42.285685+0000","last_clean":"2024-03-08T08:27:42.285685+0000","last_became_active":"2024-03-08T08:27:39.682486+0000","last_became_peered":"2024-03-08T08:27:39.682486+0000","last_unstale":"2024-03-08T08:27:42.285685+0000","last_undegraded":"2024-03-08T08:27:42.285685+0000","last_fullsized":"2024-03-08T08:27:42.285685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:01:26.479387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665493+0000","last_change":"2024-03-08T08:27:39.681720+0000","last_active":"2024-03-08T08:27:40.665493+0000","last_peered":"2024-03-08T08:27:40.665493+0000","last_clean":"2024-03-08T08:27:40.665493+0000","last_became_active":"2024-03-08T08:27:39.681473+0000","last_became_peered":"2024-03-08T08:27:39.681473+0000","last_unstale":"2024-03-08T08:27:40.665493+0000","last_undegraded":"2024-03-08T08:27:40.665493+0000","last_fullsized":"2024-03-08T08:27:40.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.681092+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.680824+0000","last_became_peered":"2024-03-08T08:27:39.680824+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665505+0000","last_change":"2024-03-08T08:27:39.674803+0000","last_active":"2024-03-08T08:27:40.665505+0000","last_peered":"2024-03-08T08:27:40.665505+0000","last_clean":"2024-03-08T08:27:40.665505+0000","last_became_active":"2024-03-08T08:27:39.674320+0000","last_became_peered":"2024-03-08T08:27:39.674320+0000","last_unstale":"2024-03-08T08:27:40.665505+0000","last_undegraded":"2024-03-08T08:27:40.665505+0000","last_fullsized":"2024-03-08T08:27:40.665505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228821+0000","last_change":"2024-03-08T08:27:39.665926+0000","last_active":"2024-03-08T08:27:43.228821+0000","last_peered":"2024-03-08T08:27:43.228821+0000","last_clean":"2024-03-08T08:27:43.228821+0000","last_became_active":"2024-03-08T08:27:39.665607+0000","last_became_peered":"2024-03-08T08:27:39.665607+0000","last_unstale":"2024-03-08T08:27:43.228821+0000","last_undegraded":"2024-03-08T08:27:43.228821+0000","last_fullsized":"2024-03-08T08:27:43.228821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229697+0000","last_change":"2024-03-08T08:27:39.666842+0000","last_active":"2024-03-08T08:27:43.229697+0000","last_peered":"2024-03-08T08:27:43.229697+0000","last_clean":"2024-03-08T08:27:43.229697+0000","last_became_active":"2024-03-08T08:27:39.666519+0000","last_became_peered":"2024-03-08T08:27:39.666519+0000","last_unstale":"2024-03-08T08:27:43.229697+0000","last_undegraded":"2024-03-08T08:27:43.229697+0000","last_fullsized":"2024-03-08T08:27:43.229697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665072+0000","last_change":"2024-03-08T08:27:39.672195+0000","last_active":"2024-03-08T08:27:40.665072+0000","last_peered":"2024-03-08T08:27:40.665072+0000","last_clean":"2024-03-08T08:27:40.665072+0000","last_became_active":"2024-03-08T08:27:39.671927+0000","last_became_peered":"2024-03-08T08:27:39.671927+0000","last_unstale":"2024-03-08T08:27:40.665072+0000","last_undegraded":"2024-03-08T08:27:40.665072+0000","last_fullsized":"2024-03-08T08:27:40.665072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229048+0000","last_change":"2024-03-08T08:27:39.677932+0000","last_active":"2024-03-08T08:27:43.229048+0000","last_peered":"2024-03-08T08:27:43.229048+0000","last_clean":"2024-03-08T08:27:43.229048+0000","last_became_active":"2024-03-08T08:27:39.674418+0000","last_became_peered":"2024-03-08T08:27:39.674418+0000","last_unstale":"2024-03-08T08:27:43.229048+0000","last_undegraded":"2024-03-08T08:27:43.229048+0000","last_fullsized":"2024-03-08T08:27:43.229048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229734+0000","last_change":"2024-03-08T08:27:39.680336+0000","last_active":"2024-03-08T08:27:43.229734+0000","last_peered":"2024-03-08T08:27:43.229734+0000","last_clean":"2024-03-08T08:27:43.229734+0000","last_became_active":"2024-03-08T08:27:39.680072+0000","last_became_peered":"2024-03-08T08:27:39.680072+0000","last_unstale":"2024-03-08T08:27:43.229734+0000","last_undegraded":"2024-03-08T08:27:43.229734+0000","last_fullsized":"2024-03-08T08:27:43.229734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:05.572283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228863+0000","last_change":"2024-03-08T08:27:39.663330+0000","last_active":"2024-03-08T08:27:43.228863+0000","last_peered":"2024-03-08T08:27:43.228863+0000","last_clean":"2024-03-08T08:27:43.228863+0000","last_became_active":"2024-03-08T08:27:39.663063+0000","last_became_peered":"2024-03-08T08:27:39.663063+0000","last_unstale":"2024-03-08T08:27:43.228863+0000","last_undegraded":"2024-03-08T08:27:43.228863+0000","last_fullsized":"2024-03-08T08:27:43.228863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285630+0000","last_change":"2024-03-08T08:27:39.673627+0000","last_active":"2024-03-08T08:27:42.285630+0000","last_peered":"2024-03-08T08:27:42.285630+0000","last_clean":"2024-03-08T08:27:42.285630+0000","last_became_active":"2024-03-08T08:27:39.673186+0000","last_became_peered":"2024-03-08T08:27:39.673186+0000","last_unstale":"2024-03-08T08:27:42.285630+0000","last_undegraded":"2024-03-08T08:27:42.285630+0000","last_fullsized":"2024-03-08T08:27:42.285630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:05.572283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664828+0000","last_change":"2024-03-08T08:27:39.672747+0000","last_active":"2024-03-08T08:27:40.664828+0000","last_peered":"2024-03-08T08:27:40.664828+0000","last_clean":"2024-03-08T08:27:40.664828+0000","last_became_active":"2024-03-08T08:27:39.672478+0000","last_became_peered":"2024-03-08T08:27:39.672478+0000","last_unstale":"2024-03-08T08:27:40.664828+0000","last_undegraded":"2024-03-08T08:27:40.664828+0000","last_fullsized":"2024-03-08T08:27:40.664828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229825+0000","last_change":"2024-03-08T08:27:39.671025+0000","last_active":"2024-03-08T08:27:43.229825+0000","last_peered":"2024-03-08T08:27:43.229825+0000","last_clean":"2024-03-08T08:27:43.229825+0000","last_became_active":"2024-03-08T08:27:39.670767+0000","last_became_peered":"2024-03-08T08:27:39.670767+0000","last_unstale":"2024-03-08T08:27:43.229825+0000","last_undegraded":"2024-03-08T08:27:43.229825+0000","last_fullsized":"2024-03-08T08:27:43.229825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229139+0000","last_change":"2024-03-08T08:27:39.680602+0000","last_active":"2024-03-08T08:27:43.229139+0000","last_peered":"2024-03-08T08:27:43.229139+0000","last_clean":"2024-03-08T08:27:43.229139+0000","last_became_active":"2024-03-08T08:27:39.679691+0000","last_became_peered":"2024-03-08T08:27:39.679691+0000","last_unstale":"2024-03-08T08:27:43.229139+0000","last_undegraded":"2024-03-08T08:27:43.229139+0000","last_fullsized":"2024-03-08T08:27:43.229139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229185+0000","last_change":"2024-03-08T08:27:39.673063+0000","last_active":"2024-03-08T08:27:43.229185+0000","last_peered":"2024-03-08T08:27:43.229185+0000","last_clean":"2024-03-08T08:27:43.229185+0000","last_became_active":"2024-03-08T08:27:39.672801+0000","last_became_peered":"2024-03-08T08:27:39.672801+0000","last_unstale":"2024-03-08T08:27:43.229185+0000","last_undegraded":"2024-03-08T08:27:43.229185+0000","last_fullsized":"2024-03-08T08:27:43.229185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229108+0000","last_change":"2024-03-08T08:27:39.680324+0000","last_active":"2024-03-08T08:27:43.229108+0000","last_peered":"2024-03-08T08:27:43.229108+0000","last_clean":"2024-03-08T08:27:43.229108+0000","last_became_active":"2024-03-08T08:27:39.679988+0000","last_became_peered":"2024-03-08T08:27:39.679988+0000","last_unstale":"2024-03-08T08:27:43.229108+0000","last_undegraded":"2024-03-08T08:27:43.229108+0000","last_fullsized":"2024-03-08T08:27:43.229108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228956+0000","last_change":"2024-03-08T08:27:39.669492+0000","last_active":"2024-03-08T08:27:43.228956+0000","last_peered":"2024-03-08T08:27:43.228956+0000","last_clean":"2024-03-08T08:27:43.228956+0000","last_became_active":"2024-03-08T08:27:39.669223+0000","last_became_peered":"2024-03-08T08:27:39.669223+0000","last_unstale":"2024-03-08T08:27:43.228956+0000","last_undegraded":"2024-03-08T08:27:43.228956+0000","last_fullsized":"2024-03-08T08:27:43.228956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665530+0000","last_change":"2024-03-08T08:27:39.675074+0000","last_active":"2024-03-08T08:27:40.665530+0000","last_peered":"2024-03-08T08:27:40.665530+0000","last_clean":"2024-03-08T08:27:40.665530+0000","last_became_active":"2024-03-08T08:27:39.674811+0000","last_became_peered":"2024-03-08T08:27:39.674811+0000","last_unstale":"2024-03-08T08:27:40.665530+0000","last_undegraded":"2024-03-08T08:27:40.665530+0000","last_fullsized":"2024-03-08T08:27:40.665530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665374+0000","last_change":"2024-03-08T08:27:39.666768+0000","last_active":"2024-03-08T08:27:40.665374+0000","last_peered":"2024-03-08T08:27:40.665374+0000","last_clean":"2024-03-08T08:27:40.665374+0000","last_became_active":"2024-03-08T08:27:39.666516+0000","last_became_peered":"2024-03-08T08:27:39.666516+0000","last_unstale":"2024-03-08T08:27:40.665374+0000","last_undegraded":"2024-03-08T08:27:40.665374+0000","last_fullsized":"2024-03-08T08:27:40.665374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229242+0000","last_change":"2024-03-08T08:27:39.672450+0000","last_active":"2024-03-08T08:27:43.229242+0000","last_peered":"2024-03-08T08:27:43.229242+0000","last_clean":"2024-03-08T08:27:43.229242+0000","last_became_active":"2024-03-08T08:27:39.671942+0000","last_became_peered":"2024-03-08T08:27:39.671942+0000","last_unstale":"2024-03-08T08:27:43.229242+0000","last_undegraded":"2024-03-08T08:27:43.229242+0000","last_fullsized":"2024-03-08T08:27:43.229242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229974+0000","last_change":"2024-03-08T08:27:39.668257+0000","last_active":"2024-03-08T08:27:43.229974+0000","last_peered":"2024-03-08T08:27:43.229974+0000","last_clean":"2024-03-08T08:27:43.229974+0000","last_became_active":"2024-03-08T08:27:39.667225+0000","last_became_peered":"2024-03-08T08:27:39.667225+0000","last_unstale":"2024-03-08T08:27:43.229974+0000","last_undegraded":"2024-03-08T08:27:43.229974+0000","last_fullsized":"2024-03-08T08:27:43.229974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229321+0000","last_change":"2024-03-08T08:27:39.680846+0000","last_active":"2024-03-08T08:27:43.229321+0000","last_peered":"2024-03-08T08:27:43.229321+0000","last_clean":"2024-03-08T08:27:43.229321+0000","last_became_active":"2024-03-08T08:27:39.680341+0000","last_became_peered":"2024-03-08T08:27:39.680341+0000","last_unstale":"2024-03-08T08:27:43.229321+0000","last_undegraded":"2024-03-08T08:27:43.229321+0000","last_fullsized":"2024-03-08T08:27:43.229321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229315+0000","last_change":"2024-03-08T08:27:39.679406+0000","last_active":"2024-03-08T08:27:43.229315+0000","last_peered":"2024-03-08T08:27:43.229315+0000","last_clean":"2024-03-08T08:27:43.229315+0000","last_became_active":"2024-03-08T08:27:39.679148+0000","last_became_peered":"2024-03-08T08:27:39.679148+0000","last_unstale":"2024-03-08T08:27:43.229315+0000","last_undegraded":"2024-03-08T08:27:43.229315+0000","last_fullsized":"2024-03-08T08:27:43.229315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664398+0000","last_change":"2024-03-08T08:27:39.670032+0000","last_active":"2024-03-08T08:27:40.664398+0000","last_peered":"2024-03-08T08:27:40.664398+0000","last_clean":"2024-03-08T08:27:40.664398+0000","last_became_active":"2024-03-08T08:27:39.669704+0000","last_became_peered":"2024-03-08T08:27:39.669704+0000","last_unstale":"2024-03-08T08:27:40.664398+0000","last_undegraded":"2024-03-08T08:27:40.664398+0000","last_fullsized":"2024-03-08T08:27:40.664398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665149+0000","last_change":"2024-03-08T08:27:39.671209+0000","last_active":"2024-03-08T08:27:40.665149+0000","last_peered":"2024-03-08T08:27:40.665149+0000","last_clean":"2024-03-08T08:27:40.665149+0000","last_became_active":"2024-03-08T08:27:39.670900+0000","last_became_peered":"2024-03-08T08:27:39.670900+0000","last_unstale":"2024-03-08T08:27:40.665149+0000","last_undegraded":"2024-03-08T08:27:40.665149+0000","last_fullsized":"2024-03-08T08:27:40.665149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229061+0000","last_change":"2024-03-08T08:27:38.661099+0000","last_active":"2024-03-08T08:27:43.229061+0000","last_peered":"2024-03-08T08:27:43.229061+0000","last_clean":"2024-03-08T08:27:43.229061+0000","last_became_active":"2024-03-08T08:27:38.660829+0000","last_became_peered":"2024-03-08T08:27:38.660829+0000","last_unstale":"2024-03-08T08:27:43.229061+0000","last_undegraded":"2024-03-08T08:27:43.229061+0000","last_fullsized":"2024-03-08T08:27:43.229061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230116+0000","last_change":"2024-03-08T08:27:39.671575+0000","last_active":"2024-03-08T08:27:43.230116+0000","last_peered":"2024-03-08T08:27:43.230116+0000","last_clean":"2024-03-08T08:27:43.230116+0000","last_became_active":"2024-03-08T08:27:39.671311+0000","last_became_peered":"2024-03-08T08:27:39.671311+0000","last_unstale":"2024-03-08T08:27:43.230116+0000","last_undegraded":"2024-03-08T08:27:43.230116+0000","last_fullsized":"2024-03-08T08:27:43.230116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664688+0000","last_change":"2024-03-08T08:27:38.651812+0000","last_active":"2024-03-08T08:27:40.664688+0000","last_peered":"2024-03-08T08:27:40.664688+0000","last_clean":"2024-03-08T08:27:40.664688+0000","last_became_active":"2024-03-08T08:27:38.651040+0000","last_became_peered":"2024-03-08T08:27:38.651040+0000","last_unstale":"2024-03-08T08:27:40.664688+0000","last_undegraded":"2024-03-08T08:27:40.664688+0000","last_fullsized":"2024-03-08T08:27:40.664688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229327+0000","last_change":"2024-03-08T08:27:38.663033+0000","last_active":"2024-03-08T08:27:43.229327+0000","last_peered":"2024-03-08T08:27:43.229327+0000","last_clean":"2024-03-08T08:27:43.229327+0000","last_became_active":"2024-03-08T08:27:38.662753+0000","last_became_peered":"2024-03-08T08:27:38.662753+0000","last_unstale":"2024-03-08T08:27:43.229327+0000","last_undegraded":"2024-03-08T08:27:43.229327+0000","last_fullsized":"2024-03-08T08:27:43.229327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:13:30.999947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228732+0000","last_change":"2024-03-08T08:27:39.670839+0000","last_active":"2024-03-08T08:27:43.228732+0000","last_peered":"2024-03-08T08:27:43.228732+0000","last_clean":"2024-03-08T08:27:43.228732+0000","last_became_active":"2024-03-08T08:27:39.670559+0000","last_became_peered":"2024-03-08T08:27:39.670559+0000","last_unstale":"2024-03-08T08:27:43.228732+0000","last_undegraded":"2024-03-08T08:27:43.228732+0000","last_fullsized":"2024-03-08T08:27:43.228732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229150+0000","last_change":"2024-03-08T08:27:38.656237+0000","last_active":"2024-03-08T08:27:43.229150+0000","last_peered":"2024-03-08T08:27:43.229150+0000","last_clean":"2024-03-08T08:27:43.229150+0000","last_became_active":"2024-03-08T08:27:38.655944+0000","last_became_peered":"2024-03-08T08:27:38.655944+0000","last_unstale":"2024-03-08T08:27:43.229150+0000","last_undegraded":"2024-03-08T08:27:43.229150+0000","last_fullsized":"2024-03-08T08:27:43.229150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.667881+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.666906+0000","last_became_peered":"2024-03-08T08:27:39.666906+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665029+0000","last_change":"2024-03-08T08:27:38.660261+0000","last_active":"2024-03-08T08:27:40.665029+0000","last_peered":"2024-03-08T08:27:40.665029+0000","last_clean":"2024-03-08T08:27:40.665029+0000","last_became_active":"2024-03-08T08:27:38.659979+0000","last_became_peered":"2024-03-08T08:27:38.659979+0000","last_unstale":"2024-03-08T08:27:40.665029+0000","last_undegraded":"2024-03-08T08:27:40.665029+0000","last_fullsized":"2024-03-08T08:27:40.665029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229622+0000","last_change":"2024-03-08T08:27:39.679621+0000","last_active":"2024-03-08T08:27:43.229622+0000","last_peered":"2024-03-08T08:27:43.229622+0000","last_clean":"2024-03-08T08:27:43.229622+0000","last_became_active":"2024-03-08T08:27:39.679327+0000","last_became_peered":"2024-03-08T08:27:39.679327+0000","last_unstale":"2024-03-08T08:27:43.229622+0000","last_undegraded":"2024-03-08T08:27:43.229622+0000","last_fullsized":"2024-03-08T08:27:43.229622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:07.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229731+0000","last_change":"2024-03-08T08:27:38.662435+0000","last_active":"2024-03-08T08:27:43.229731+0000","last_peered":"2024-03-08T08:27:43.229731+0000","last_clean":"2024-03-08T08:27:43.229731+0000","last_became_active":"2024-03-08T08:27:38.662189+0000","last_became_peered":"2024-03-08T08:27:38.662189+0000","last_unstale":"2024-03-08T08:27:43.229731+0000","last_undegraded":"2024-03-08T08:27:43.229731+0000","last_fullsized":"2024-03-08T08:27:43.229731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230094+0000","last_change":"2024-03-08T08:27:39.680354+0000","last_active":"2024-03-08T08:27:43.230094+0000","last_peered":"2024-03-08T08:27:43.230094+0000","last_clean":"2024-03-08T08:27:43.230094+0000","last_became_active":"2024-03-08T08:27:39.680068+0000","last_became_peered":"2024-03-08T08:27:39.680068+0000","last_unstale":"2024-03-08T08:27:43.230094+0000","last_undegraded":"2024-03-08T08:27:43.230094+0000","last_fullsized":"2024-03-08T08:27:43.230094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665344+0000","last_change":"2024-03-08T08:27:38.662811+0000","last_active":"2024-03-08T08:27:40.665344+0000","last_peered":"2024-03-08T08:27:40.665344+0000","last_clean":"2024-03-08T08:27:40.665344+0000","last_became_active":"2024-03-08T08:27:38.662484+0000","last_became_peered":"2024-03-08T08:27:38.662484+0000","last_unstale":"2024-03-08T08:27:40.665344+0000","last_undegraded":"2024-03-08T08:27:40.665344+0000","last_fullsized":"2024-03-08T08:27:40.665344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665494+0000","last_change":"2024-03-08T08:27:39.681653+0000","last_active":"2024-03-08T08:27:40.665494+0000","last_peered":"2024-03-08T08:27:40.665494+0000","last_clean":"2024-03-08T08:27:40.665494+0000","last_became_active":"2024-03-08T08:27:39.681368+0000","last_became_peered":"2024-03-08T08:27:39.681368+0000","last_unstale":"2024-03-08T08:27:40.665494+0000","last_undegraded":"2024-03-08T08:27:40.665494+0000","last_fullsized":"2024-03-08T08:27:40.665494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664972+0000","last_change":"2024-03-08T08:27:38.655367+0000","last_active":"2024-03-08T08:27:40.664972+0000","last_peered":"2024-03-08T08:27:40.664972+0000","last_clean":"2024-03-08T08:27:40.664972+0000","last_became_active":"2024-03-08T08:27:38.655051+0000","last_became_peered":"2024-03-08T08:27:38.655051+0000","last_unstale":"2024-03-08T08:27:40.664972+0000","last_undegraded":"2024-03-08T08:27:40.664972+0000","last_fullsized":"2024-03-08T08:27:40.664972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285287+0000","last_change":"2024-03-08T08:27:39.672887+0000","last_active":"2024-03-08T08:27:42.285287+0000","last_peered":"2024-03-08T08:27:42.285287+0000","last_clean":"2024-03-08T08:27:42.285287+0000","last_became_active":"2024-03-08T08:27:39.672009+0000","last_became_peered":"2024-03-08T08:27:39.672009+0000","last_unstale":"2024-03-08T08:27:42.285287+0000","last_undegraded":"2024-03-08T08:27:42.285287+0000","last_fullsized":"2024-03-08T08:27:42.285287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230150+0000","last_change":"2024-03-08T08:27:38.650529+0000","last_active":"2024-03-08T08:27:43.230150+0000","last_peered":"2024-03-08T08:27:43.230150+0000","last_clean":"2024-03-08T08:27:43.230150+0000","last_became_active":"2024-03-08T08:27:38.650183+0000","last_became_peered":"2024-03-08T08:27:38.650183+0000","last_unstale":"2024-03-08T08:27:43.230150+0000","last_undegraded":"2024-03-08T08:27:43.230150+0000","last_fullsized":"2024-03-08T08:27:43.230150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:18:22.093931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229702+0000","last_change":"2024-03-08T08:27:39.670409+0000","last_active":"2024-03-08T08:27:43.229702+0000","last_peered":"2024-03-08T08:27:43.229702+0000","last_clean":"2024-03-08T08:27:43.229702+0000","last_became_active":"2024-03-08T08:27:39.670105+0000","last_became_peered":"2024-03-08T08:27:39.670105+0000","last_unstale":"2024-03-08T08:27:43.229702+0000","last_undegraded":"2024-03-08T08:27:43.229702+0000","last_fullsized":"2024-03-08T08:27:43.229702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229241+0000","last_change":"2024-03-08T08:27:38.660419+0000","last_active":"2024-03-08T08:27:43.229241+0000","last_peered":"2024-03-08T08:27:43.229241+0000","last_clean":"2024-03-08T08:27:43.229241+0000","last_became_active":"2024-03-08T08:27:38.660225+0000","last_became_peered":"2024-03-08T08:27:38.660225+0000","last_unstale":"2024-03-08T08:27:43.229241+0000","last_undegraded":"2024-03-08T08:27:43.229241+0000","last_fullsized":"2024-03-08T08:27:43.229241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285354+0000","last_change":"2024-03-08T08:27:39.675352+0000","last_active":"2024-03-08T08:27:42.285354+0000","last_peered":"2024-03-08T08:27:42.285354+0000","last_clean":"2024-03-08T08:27:42.285354+0000","last_became_active":"2024-03-08T08:27:39.674881+0000","last_became_peered":"2024-03-08T08:27:39.674881+0000","last_unstale":"2024-03-08T08:27:42.285354+0000","last_undegraded":"2024-03-08T08:27:42.285354+0000","last_fullsized":"2024-03-08T08:27:42.285354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664467+0000","last_change":"2024-03-08T08:27:38.656182+0000","last_active":"2024-03-08T08:27:40.664467+0000","last_peered":"2024-03-08T08:27:40.664467+0000","last_clean":"2024-03-08T08:27:40.664467+0000","last_became_active":"2024-03-08T08:27:38.655315+0000","last_became_peered":"2024-03-08T08:27:38.655315+0000","last_unstale":"2024-03-08T08:27:40.664467+0000","last_undegraded":"2024-03-08T08:27:40.664467+0000","last_fullsized":"2024-03-08T08:27:40.664467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664532+0000","last_change":"2024-03-08T08:27:39.666898+0000","last_active":"2024-03-08T08:27:40.664532+0000","last_peered":"2024-03-08T08:27:40.664532+0000","last_clean":"2024-03-08T08:27:40.664532+0000","last_became_active":"2024-03-08T08:27:39.666563+0000","last_became_peered":"2024-03-08T08:27:39.666563+0000","last_unstale":"2024-03-08T08:27:40.664532+0000","last_undegraded":"2024-03-08T08:27:40.664532+0000","last_fullsized":"2024-03-08T08:27:40.664532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229594+0000","last_change":"2024-03-08T08:27:38.656580+0000","last_active":"2024-03-08T08:27:43.229594+0000","last_peered":"2024-03-08T08:27:43.229594+0000","last_clean":"2024-03-08T08:27:43.229594+0000","last_became_active":"2024-03-08T08:27:38.656233+0000","last_became_peered":"2024-03-08T08:27:38.656233+0000","last_unstale":"2024-03-08T08:27:43.229594+0000","last_undegraded":"2024-03-08T08:27:43.229594+0000","last_fullsized":"2024-03-08T08:27:43.229594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285501+0000","last_change":"2024-03-08T08:27:39.677499+0000","last_active":"2024-03-08T08:27:42.285501+0000","last_peered":"2024-03-08T08:27:42.285501+0000","last_clean":"2024-03-08T08:27:42.285501+0000","last_became_active":"2024-03-08T08:27:39.677213+0000","last_became_peered":"2024-03-08T08:27:39.677213+0000","last_unstale":"2024-03-08T08:27:42.285501+0000","last_undegraded":"2024-03-08T08:27:42.285501+0000","last_fullsized":"2024-03-08T08:27:42.285501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:00.627737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665133+0000","last_change":"2024-03-08T08:27:38.653100+0000","last_active":"2024-03-08T08:27:40.665133+0000","last_peered":"2024-03-08T08:27:40.665133+0000","last_clean":"2024-03-08T08:27:40.665133+0000","last_became_active":"2024-03-08T08:27:38.652837+0000","last_became_peered":"2024-03-08T08:27:38.652837+0000","last_unstale":"2024-03-08T08:27:40.665133+0000","last_undegraded":"2024-03-08T08:27:40.665133+0000","last_fullsized":"2024-03-08T08:27:40.665133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285597+0000","last_change":"2024-03-08T08:27:39.673698+0000","last_active":"2024-03-08T08:27:42.285597+0000","last_peered":"2024-03-08T08:27:42.285597+0000","last_clean":"2024-03-08T08:27:42.285597+0000","last_became_active":"2024-03-08T08:27:39.673424+0000","last_became_peered":"2024-03-08T08:27:39.673424+0000","last_unstale":"2024-03-08T08:27:42.285597+0000","last_undegraded":"2024-03-08T08:27:42.285597+0000","last_fullsized":"2024-03-08T08:27:42.285597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:12:43.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228967+0000","last_change":"2024-03-08T08:27:38.660277+0000","last_active":"2024-03-08T08:27:43.228967+0000","last_peered":"2024-03-08T08:27:43.228967+0000","last_clean":"2024-03-08T08:27:43.228967+0000","last_became_active":"2024-03-08T08:27:38.659873+0000","last_became_peered":"2024-03-08T08:27:38.659873+0000","last_unstale":"2024-03-08T08:27:43.228967+0000","last_undegraded":"2024-03-08T08:27:43.228967+0000","last_fullsized":"2024-03-08T08:27:43.228967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285694+0000","last_change":"2024-03-08T08:27:39.674403+0000","last_active":"2024-03-08T08:27:42.285694+0000","last_peered":"2024-03-08T08:27:42.285694+0000","last_clean":"2024-03-08T08:27:42.285694+0000","last_became_active":"2024-03-08T08:27:39.674152+0000","last_became_peered":"2024-03-08T08:27:39.674152+0000","last_unstale":"2024-03-08T08:27:42.285694+0000","last_undegraded":"2024-03-08T08:27:42.285694+0000","last_fullsized":"2024-03-08T08:27:42.285694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:58:36.475015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665435+0000","last_change":"2024-03-08T08:27:38.655005+0000","last_active":"2024-03-08T08:27:40.665435+0000","last_peered":"2024-03-08T08:27:40.665435+0000","last_clean":"2024-03-08T08:27:40.665435+0000","last_became_active":"2024-03-08T08:27:38.654683+0000","last_became_peered":"2024-03-08T08:27:38.654683+0000","last_unstale":"2024-03-08T08:27:40.665435+0000","last_undegraded":"2024-03-08T08:27:40.665435+0000","last_fullsized":"2024-03-08T08:27:40.665435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229535+0000","last_change":"2024-03-08T08:27:39.673959+0000","last_active":"2024-03-08T08:27:43.229535+0000","last_peered":"2024-03-08T08:27:43.229535+0000","last_clean":"2024-03-08T08:27:43.229535+0000","last_became_active":"2024-03-08T08:27:39.673369+0000","last_became_peered":"2024-03-08T08:27:39.673369+0000","last_unstale":"2024-03-08T08:27:43.229535+0000","last_undegraded":"2024-03-08T08:27:43.229535+0000","last_fullsized":"2024-03-08T08:27:43.229535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:00.627737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229488+0000","last_change":"2024-03-08T08:27:38.661161+0000","last_active":"2024-03-08T08:27:43.229488+0000","last_peered":"2024-03-08T08:27:43.229488+0000","last_clean":"2024-03-08T08:27:43.229488+0000","last_became_active":"2024-03-08T08:27:38.660876+0000","last_became_peered":"2024-03-08T08:27:38.660876+0000","last_unstale":"2024-03-08T08:27:43.229488+0000","last_undegraded":"2024-03-08T08:27:43.229488+0000","last_fullsized":"2024-03-08T08:27:43.229488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285472+0000","last_change":"2024-03-08T08:27:39.682454+0000","last_active":"2024-03-08T08:27:42.285472+0000","last_peered":"2024-03-08T08:27:42.285472+0000","last_clean":"2024-03-08T08:27:42.285472+0000","last_became_active":"2024-03-08T08:27:39.682178+0000","last_became_peered":"2024-03-08T08:27:39.682178+0000","last_unstale":"2024-03-08T08:27:42.285472+0000","last_undegraded":"2024-03-08T08:27:42.285472+0000","last_fullsized":"2024-03-08T08:27:42.285472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:07.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285074+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:27:42.285074+0000","last_peered":"2024-03-08T08:27:42.285074+0000","last_clean":"2024-03-08T08:27:42.285074+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:27:42.285074+0000","last_undegraded":"2024-03-08T08:27:42.285074+0000","last_fullsized":"2024-03-08T08:27:42.285074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229052+0000","last_change":"2024-03-08T08:27:38.659294+0000","last_active":"2024-03-08T08:27:43.229052+0000","last_peered":"2024-03-08T08:27:43.229052+0000","last_clean":"2024-03-08T08:27:43.229052+0000","last_became_active":"2024-03-08T08:27:38.659037+0000","last_became_peered":"2024-03-08T08:27:38.659037+0000","last_unstale":"2024-03-08T08:27:43.229052+0000","last_undegraded":"2024-03-08T08:27:43.229052+0000","last_fullsized":"2024-03-08T08:27:43.229052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228947+0000","last_change":"2024-03-08T08:27:39.681265+0000","last_active":"2024-03-08T08:27:43.228947+0000","last_peered":"2024-03-08T08:27:43.228947+0000","last_clean":"2024-03-08T08:27:43.228947+0000","last_became_active":"2024-03-08T08:27:39.680995+0000","last_became_peered":"2024-03-08T08:27:39.680995+0000","last_unstale":"2024-03-08T08:27:43.228947+0000","last_undegraded":"2024-03-08T08:27:43.228947+0000","last_fullsized":"2024-03-08T08:27:43.228947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.284946+0000","last_change":"2024-03-08T08:27:38.663815+0000","last_active":"2024-03-08T08:27:42.284946+0000","last_peered":"2024-03-08T08:27:42.284946+0000","last_clean":"2024-03-08T08:27:42.284946+0000","last_became_active":"2024-03-08T08:27:38.663589+0000","last_became_peered":"2024-03-08T08:27:38.663589+0000","last_unstale":"2024-03-08T08:27:42.284946+0000","last_undegraded":"2024-03-08T08:27:42.284946+0000","last_fullsized":"2024-03-08T08:27:42.284946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229606+0000","last_change":"2024-03-08T08:27:39.673596+0000","last_active":"2024-03-08T08:27:43.229606+0000","last_peered":"2024-03-08T08:27:43.229606+0000","last_clean":"2024-03-08T08:27:43.229606+0000","last_became_active":"2024-03-08T08:27:39.673171+0000","last_became_peered":"2024-03-08T08:27:39.673171+0000","last_unstale":"2024-03-08T08:27:43.229606+0000","last_undegraded":"2024-03-08T08:27:43.229606+0000","last_fullsized":"2024-03-08T08:27:43.229606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229195+0000","last_change":"2024-03-08T08:27:38.658281+0000","last_active":"2024-03-08T08:27:43.229195+0000","last_peered":"2024-03-08T08:27:43.229195+0000","last_clean":"2024-03-08T08:27:43.229195+0000","last_became_active":"2024-03-08T08:27:38.657983+0000","last_became_peered":"2024-03-08T08:27:38.657983+0000","last_unstale":"2024-03-08T08:27:43.229195+0000","last_undegraded":"2024-03-08T08:27:43.229195+0000","last_fullsized":"2024-03-08T08:27:43.229195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.662315+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.661362+0000","last_became_peered":"2024-03-08T08:27:39.661362+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229349+0000","last_change":"2024-03-08T08:27:38.659980+0000","last_active":"2024-03-08T08:27:43.229349+0000","last_peered":"2024-03-08T08:27:43.229349+0000","last_clean":"2024-03-08T08:27:43.229349+0000","last_became_active":"2024-03-08T08:27:38.659704+0000","last_became_peered":"2024-03-08T08:27:38.659704+0000","last_unstale":"2024-03-08T08:27:43.229349+0000","last_undegraded":"2024-03-08T08:27:43.229349+0000","last_fullsized":"2024-03-08T08:27:43.229349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665238+0000","last_change":"2024-03-08T08:27:39.678619+0000","last_active":"2024-03-08T08:27:40.665238+0000","last_peered":"2024-03-08T08:27:40.665238+0000","last_clean":"2024-03-08T08:27:40.665238+0000","last_became_active":"2024-03-08T08:27:39.678371+0000","last_became_peered":"2024-03-08T08:27:39.678371+0000","last_unstale":"2024-03-08T08:27:40.665238+0000","last_undegraded":"2024-03-08T08:27:40.665238+0000","last_fullsized":"2024-03-08T08:27:40.665238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665304+0000","last_change":"2024-03-08T08:27:38.654132+0000","last_active":"2024-03-08T08:27:40.665304+0000","last_peered":"2024-03-08T08:27:40.665304+0000","last_clean":"2024-03-08T08:27:40.665304+0000","last_became_active":"2024-03-08T08:27:38.653844+0000","last_became_peered":"2024-03-08T08:27:38.653844+0000","last_unstale":"2024-03-08T08:27:40.665304+0000","last_undegraded":"2024-03-08T08:27:40.665304+0000","last_fullsized":"2024-03-08T08:27:40.665304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229651+0000","last_change":"2024-03-08T08:27:39.679299+0000","last_active":"2024-03-08T08:27:43.229651+0000","last_peered":"2024-03-08T08:27:43.229651+0000","last_clean":"2024-03-08T08:27:43.229651+0000","last_became_active":"2024-03-08T08:27:39.679023+0000","last_became_peered":"2024-03-08T08:27:39.679023+0000","last_unstale":"2024-03-08T08:27:43.229651+0000","last_undegraded":"2024-03-08T08:27:43.229651+0000","last_fullsized":"2024-03-08T08:27:43.229651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:12:43.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285620+0000","last_change":"2024-03-08T08:27:38.665656+0000","last_active":"2024-03-08T08:27:42.285620+0000","last_peered":"2024-03-08T08:27:42.285620+0000","last_clean":"2024-03-08T08:27:42.285620+0000","last_became_active":"2024-03-08T08:27:38.665370+0000","last_became_peered":"2024-03-08T08:27:38.665370+0000","last_unstale":"2024-03-08T08:27:42.285620+0000","last_undegraded":"2024-03-08T08:27:42.285620+0000","last_fullsized":"2024-03-08T08:27:42.285620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665354+0000","last_change":"2024-03-08T08:27:39.664052+0000","last_active":"2024-03-08T08:27:40.665354+0000","last_peered":"2024-03-08T08:27:40.665354+0000","last_clean":"2024-03-08T08:27:40.665354+0000","last_became_active":"2024-03-08T08:27:39.663794+0000","last_became_peered":"2024-03-08T08:27:39.663794+0000","last_unstale":"2024-03-08T08:27:40.665354+0000","last_undegraded":"2024-03-08T08:27:40.665354+0000","last_fullsized":"2024-03-08T08:27:40.665354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:41.113731+0000","last_change":"2024-03-08T08:27:38.658214+0000","last_active":"2024-03-08T08:27:41.113731+0000","last_peered":"2024-03-08T08:27:41.113731+0000","last_clean":"2024-03-08T08:27:41.113731+0000","last_became_active":"2024-03-08T08:27:38.657944+0000","last_became_peered":"2024-03-08T08:27:38.657944+0000","last_unstale":"2024-03-08T08:27:41.113731+0000","last_undegraded":"2024-03-08T08:27:41.113731+0000","last_fullsized":"2024-03-08T08:27:41.113731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229036+0000","last_change":"2024-03-08T08:27:39.671410+0000","last_active":"2024-03-08T08:27:43.229036+0000","last_peered":"2024-03-08T08:27:43.229036+0000","last_clean":"2024-03-08T08:27:43.229036+0000","last_became_active":"2024-03-08T08:27:39.671169+0000","last_became_peered":"2024-03-08T08:27:39.671169+0000","last_unstale":"2024-03-08T08:27:43.229036+0000","last_undegraded":"2024-03-08T08:27:43.229036+0000","last_fullsized":"2024-03-08T08:27:43.229036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665140+0000","last_change":"2024-03-08T08:27:38.657833+0000","last_active":"2024-03-08T08:27:40.665140+0000","last_peered":"2024-03-08T08:27:40.665140+0000","last_clean":"2024-03-08T08:27:40.665140+0000","last_became_active":"2024-03-08T08:27:38.657560+0000","last_became_peered":"2024-03-08T08:27:38.657560+0000","last_unstale":"2024-03-08T08:27:40.665140+0000","last_undegraded":"2024-03-08T08:27:40.665140+0000","last_fullsized":"2024-03-08T08:27:40.665140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230104+0000","last_change":"2024-03-08T08:27:39.668821+0000","last_active":"2024-03-08T08:27:43.230104+0000","last_peered":"2024-03-08T08:27:43.230104+0000","last_clean":"2024-03-08T08:27:43.230104+0000","last_became_active":"2024-03-08T08:27:39.668559+0000","last_became_peered":"2024-03-08T08:27:39.668559+0000","last_unstale":"2024-03-08T08:27:43.230104+0000","last_undegraded":"2024-03-08T08:27:43.230104+0000","last_fullsized":"2024-03-08T08:27:43.230104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.284921+0000","last_change":"2024-03-08T08:27:38.665148+0000","last_active":"2024-03-08T08:27:42.284921+0000","last_peered":"2024-03-08T08:27:42.284921+0000","last_clean":"2024-03-08T08:27:42.284921+0000","last_became_active":"2024-03-08T08:27:38.664834+0000","last_became_peered":"2024-03-08T08:27:38.664834+0000","last_unstale":"2024-03-08T08:27:42.284921+0000","last_undegraded":"2024-03-08T08:27:42.284921+0000","last_fullsized":"2024-03-08T08:27:42.284921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.667641+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.667296+0000","last_became_peered":"2024-03-08T08:27:39.667296+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665163+0000","last_change":"2024-03-08T08:27:38.660688+0000","last_active":"2024-03-08T08:27:40.665163+0000","last_peered":"2024-03-08T08:27:40.665163+0000","last_clean":"2024-03-08T08:27:40.665163+0000","last_became_active":"2024-03-08T08:27:38.660427+0000","last_became_peered":"2024-03-08T08:27:38.660427+0000","last_unstale":"2024-03-08T08:27:40.665163+0000","last_undegraded":"2024-03-08T08:27:40.665163+0000","last_fullsized":"2024-03-08T08:27:40.665163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228902+0000","last_change":"2024-03-08T08:27:39.665202+0000","last_active":"2024-03-08T08:27:43.228902+0000","last_peered":"2024-03-08T08:27:43.228902+0000","last_clean":"2024-03-08T08:27:43.228902+0000","last_became_active":"2024-03-08T08:27:39.663567+0000","last_became_peered":"2024-03-08T08:27:39.663567+0000","last_unstale":"2024-03-08T08:27:43.228902+0000","last_undegraded":"2024-03-08T08:27:43.228902+0000","last_fullsized":"2024-03-08T08:27:43.228902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665293+0000","last_change":"2024-03-08T08:27:38.660223+0000","last_active":"2024-03-08T08:27:40.665293+0000","last_peered":"2024-03-08T08:27:40.665293+0000","last_clean":"2024-03-08T08:27:40.665293+0000","last_became_active":"2024-03-08T08:27:38.659939+0000","last_became_peered":"2024-03-08T08:27:38.659939+0000","last_unstale":"2024-03-08T08:27:40.665293+0000","last_undegraded":"2024-03-08T08:27:40.665293+0000","last_fullsized":"2024-03-08T08:27:40.665293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228969+0000","last_change":"2024-03-08T08:27:39.670191+0000","last_active":"2024-03-08T08:27:43.228969+0000","last_peered":"2024-03-08T08:27:43.228969+0000","last_clean":"2024-03-08T08:27:43.228969+0000","last_became_active":"2024-03-08T08:27:39.669921+0000","last_became_peered":"2024-03-08T08:27:39.669921+0000","last_unstale":"2024-03-08T08:27:43.228969+0000","last_undegraded":"2024-03-08T08:27:43.228969+0000","last_fullsized":"2024-03-08T08:27:43.228969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285389+0000","last_change":"2024-03-08T08:27:38.665118+0000","last_active":"2024-03-08T08:27:42.285389+0000","last_peered":"2024-03-08T08:27:42.285389+0000","last_clean":"2024-03-08T08:27:42.285389+0000","last_became_active":"2024-03-08T08:27:38.664812+0000","last_became_peered":"2024-03-08T08:27:38.664812+0000","last_unstale":"2024-03-08T08:27:42.285389+0000","last_undegraded":"2024-03-08T08:27:42.285389+0000","last_fullsized":"2024-03-08T08:27:42.285389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664566+0000","last_change":"2024-03-08T08:27:39.672103+0000","last_active":"2024-03-08T08:27:40.664566+0000","last_peered":"2024-03-08T08:27:40.664566+0000","last_clean":"2024-03-08T08:27:40.664566+0000","last_became_active":"2024-03-08T08:27:39.671821+0000","last_became_peered":"2024-03-08T08:27:39.671821+0000","last_unstale":"2024-03-08T08:27:40.664566+0000","last_undegraded":"2024-03-08T08:27:40.664566+0000","last_fullsized":"2024-03-08T08:27:40.664566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229542+0000","last_change":"2024-03-08T08:27:39.676821+0000","last_active":"2024-03-08T08:27:43.229542+0000","last_peered":"2024-03-08T08:27:43.229542+0000","last_clean":"2024-03-08T08:27:43.229542+0000","last_became_active":"2024-03-08T08:27:39.674571+0000","last_became_peered":"2024-03-08T08:27:39.674571+0000","last_unstale":"2024-03-08T08:27:43.229542+0000","last_undegraded":"2024-03-08T08:27:43.229542+0000","last_fullsized":"2024-03-08T08:27:43.229542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:58.646243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285171+0000","last_change":"2024-03-08T08:27:38.667618+0000","last_active":"2024-03-08T08:27:42.285171+0000","last_peered":"2024-03-08T08:27:42.285171+0000","last_clean":"2024-03-08T08:27:42.285171+0000","last_became_active":"2024-03-08T08:27:38.667361+0000","last_became_peered":"2024-03-08T08:27:38.667361+0000","last_unstale":"2024-03-08T08:27:42.285171+0000","last_undegraded":"2024-03-08T08:27:42.285171+0000","last_fullsized":"2024-03-08T08:27:42.285171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229446+0000","last_change":"2024-03-08T08:27:39.679501+0000","last_active":"2024-03-08T08:27:43.229446+0000","last_peered":"2024-03-08T08:27:43.229446+0000","last_clean":"2024-03-08T08:27:43.229446+0000","last_became_active":"2024-03-08T08:27:39.679204+0000","last_became_peered":"2024-03-08T08:27:39.679204+0000","last_unstale":"2024-03-08T08:27:43.229446+0000","last_undegraded":"2024-03-08T08:27:43.229446+0000","last_fullsized":"2024-03-08T08:27:43.229446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285442+0000","last_change":"2024-03-08T08:27:38.667616+0000","last_active":"2024-03-08T08:27:42.285442+0000","last_peered":"2024-03-08T08:27:42.285442+0000","last_clean":"2024-03-08T08:27:42.285442+0000","last_became_active":"2024-03-08T08:27:38.667335+0000","last_became_peered":"2024-03-08T08:27:38.667335+0000","last_unstale":"2024-03-08T08:27:42.285442+0000","last_undegraded":"2024-03-08T08:27:42.285442+0000","last_fullsized":"2024-03-08T08:27:42.285442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229948+0000","last_change":"2024-03-08T08:27:39.681042+0000","last_active":"2024-03-08T08:27:43.229948+0000","last_peered":"2024-03-08T08:27:43.229948+0000","last_clean":"2024-03-08T08:27:43.229948+0000","last_became_active":"2024-03-08T08:27:39.680791+0000","last_became_peered":"2024-03-08T08:27:39.680791+0000","last_unstale":"2024-03-08T08:27:43.229948+0000","last_undegraded":"2024-03-08T08:27:43.229948+0000","last_fullsized":"2024-03-08T08:27:43.229948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229414+0000","last_change":"2024-03-08T08:27:38.659145+0000","last_active":"2024-03-08T08:27:43.229414+0000","last_peered":"2024-03-08T08:27:43.229414+0000","last_clean":"2024-03-08T08:27:43.229414+0000","last_became_active":"2024-03-08T08:27:38.658848+0000","last_became_peered":"2024-03-08T08:27:38.658848+0000","last_unstale":"2024-03-08T08:27:43.229414+0000","last_undegraded":"2024-03-08T08:27:43.229414+0000","last_fullsized":"2024-03-08T08:27:43.229414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229123+0000","last_change":"2024-03-08T08:27:39.668041+0000","last_active":"2024-03-08T08:27:43.229123+0000","last_peered":"2024-03-08T08:27:43.229123+0000","last_clean":"2024-03-08T08:27:43.229123+0000","last_became_active":"2024-03-08T08:27:39.667724+0000","last_became_peered":"2024-03-08T08:27:39.667724+0000","last_unstale":"2024-03-08T08:27:43.229123+0000","last_undegraded":"2024-03-08T08:27:43.229123+0000","last_fullsized":"2024-03-08T08:27:43.229123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230114+0000","last_change":"2024-03-08T08:27:38.657913+0000","last_active":"2024-03-08T08:27:43.230114+0000","last_peered":"2024-03-08T08:27:43.230114+0000","last_clean":"2024-03-08T08:27:43.230114+0000","last_became_active":"2024-03-08T08:27:38.657593+0000","last_became_peered":"2024-03-08T08:27:38.657593+0000","last_unstale":"2024-03-08T08:27:43.230114+0000","last_undegraded":"2024-03-08T08:27:43.230114+0000","last_fullsized":"2024-03-08T08:27:43.230114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:27:51.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:27:51.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.400+0000 7fe464ba1640 1 Processor -- start 2024-03-08T08:27:51.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.400+0000 7fe464ba1640 1 -- start start 2024-03-08T08:27:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe464ba1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe46015dfa0 0x7fe46006c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe464ba1640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe464ba1640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46015e380 con 0x7fe46006b040 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe464ba1640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4600b5f60 con 0x7fe4600ad910 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe464ba1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4600b6240 con 0x7fe46015dfa0 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe45e59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe46015dfa0 0x7fe46006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe45ed9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe45dd9b640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe46006b040 0x7fe4600ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:40978/0 (socket says 172.21.15.105:40978) 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.401+0000 7fe45dd9b640 1 -- 172.21.15.105:0/1509481058 learned_addr learned my addr 172.21.15.105:0/1509481058 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.402+0000 7fe45e59c640 1 -- 172.21.15.105:0/1509481058 >> v2:172.21.15.144:3300/0 conn(0x7fe4600ad910 msgr2=0x7fe4600d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.402+0000 7fe45e59c640 1 --2- 172.21.15.105:0/1509481058 >> v2:172.21.15.144:3300/0 conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.402+0000 7fe45e59c640 1 -- 172.21.15.105:0/1509481058 >> v1:172.21.15.144:6789/0 conn(0x7fe46006b040 legacy=0x7fe4600ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.402+0000 7fe45d59a640 1 -- 172.21.15.105:0/1509481058 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064208372 0 0) 0x7fe46015e380 con 0x7fe46006b040 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.402+0000 7fe45e59c640 1 -- 172.21.15.105:0/1509481058 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46006b720 con 0x7fe46015dfa0 2024-03-08T08:27:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.402+0000 7fe45ed9d640 1 --2- 172.21.15.105:0/1509481058 >> v2:172.21.15.144:3300/0 conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:27:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.402+0000 7fe45e59c640 1 --2- 172.21.15.105:0/1509481058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe46015dfa0 0x7fe46006c200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097a0 tx=0x7fe45002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60929b9d7003fdaf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe45d59a640 1 -- 172.21.15.105:0/1509481058 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450035020 con 0x7fe46015dfa0 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe45d59a640 1 -- 172.21.15.105:0/1509481058 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450031830 con 0x7fe46015dfa0 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe45d59a640 1 -- 172.21.15.105:0/1509481058 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450031c00 con 0x7fe46015dfa0 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe464ba1640 1 -- 172.21.15.105:0/1509481058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe46015dfa0 msgr2=0x7fe46006c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1509481058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe46015dfa0 0x7fe46006c200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097a0 tx=0x7fe45002b8d0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe464ba1640 1 -- 172.21.15.105:0/1509481058 shutdown_connections 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1509481058 >> v2:172.21.15.144:3300/0 conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1509481058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe46015dfa0 0x7fe46006c200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe464ba1640 1 -- 172.21.15.105:0/1509481058 >> 172.21.15.105:0/1509481058 conn(0x7fe4600b50f0 msgr2=0x7fe4600b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe464ba1640 1 -- 172.21.15.105:0/1509481058 shutdown_connections 2024-03-08T08:27:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.403+0000 7fe464ba1640 1 -- 172.21.15.105:0/1509481058 wait complete. 2024-03-08T08:27:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe464ba1640 1 Processor -- start 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe464ba1640 1 -- start start 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe464ba1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46006b040 0x7fe460157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe464ba1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe464ba1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46015dfa0 0x7fe4600d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe464ba1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46006d000 con 0x7fe46006b040 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe464ba1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46006bce0 con 0x7fe46015dfa0 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe464ba1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46006d180 con 0x7fe4600ad910 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe45e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46006b040 0x7fe460157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe45e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46006b040 0x7fe460157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59726/0 (socket says 172.21.15.105:59726) 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.405+0000 7fe45e59c640 1 -- 172.21.15.105:0/1286160452 learned_addr learned my addr 172.21.15.105:0/1286160452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45dd9b640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45ed9d640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46015dfa0 0x7fe4600d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45e59c640 1 -- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4600ad910 msgr2=0x7fe4600d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45e59c640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45e59c640 1 -- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46015dfa0 msgr2=0x7fe4600d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45e59c640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46015dfa0 0x7fe4600d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45e59c640 1 -- 172.21.15.105:0/1286160452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46009d4c0 con 0x7fe46006b040 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45ed9d640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46015dfa0 0x7fe4600d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe45e59c640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46006b040 0x7fe460157660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097a0 tx=0x7fe4500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe44effd640 1 -- 172.21.15.105:0/1286160452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450004230 con 0x7fe46006b040 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe44effd640 1 -- 172.21.15.105:0/1286160452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4500043f0 con 0x7fe46006b040 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46009d760 con 0x7fe46006b040 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe44effd640 1 -- 172.21.15.105:0/1286160452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45003a870 con 0x7fe46006b040 2024-03-08T08:27:51.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.406+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46009bd80 con 0x7fe46006b040 2024-03-08T08:27:51.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.407+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe460075120 con 0x7fe46006b040 2024-03-08T08:27:51.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.407+0000 7fe44effd640 1 -- 172.21.15.105:0/1286160452 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe450038020 con 0x7fe46006b040 2024-03-08T08:27:51.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.408+0000 7fe44effd640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe44006b500 0x7fe44006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.408+0000 7fe45dd9b640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe44006b500 0x7fe44006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.408+0000 7fe44effd640 1 -- 172.21.15.105:0/1286160452 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4500abe00 con 0x7fe46006b040 2024-03-08T08:27:51.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.408+0000 7fe45dd9b640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe44006b500 0x7fe44006d9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe46009a980 tx=0x7fe454007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.413+0000 7fe44effd640 1 -- 172.21.15.105:0/1286160452 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe450040070 con 0x7fe46006b040 2024-03-08T08:27:51.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.581+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe460071ae0 con 0x7fe44006b500 2024-03-08T08:27:51.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.586+0000 7fe44effd640 1 -- 172.21.15.105:0/1286160452 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251837 (secure 0 0 0) 0x7fe460071ae0 con 0x7fe44006b500 2024-03-08T08:27:51.592 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:51.593 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:27:51.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe44006b500 msgr2=0x7fe44006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe44006b500 0x7fe44006d9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe46009a980 tx=0x7fe454007d50 comp rx=0 tx=0).stop 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46006b040 msgr2=0x7fe460157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46006b040 0x7fe460157660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097a0 tx=0x7fe4500046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 shutdown_connections 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe44006b500 0x7fe44006d9c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe46015dfa0 0x7fe4600d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4600ad910 0x7fe4600d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 --2- 172.21.15.105:0/1286160452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe46006b040 0x7fe460157660 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 >> 172.21.15.105:0/1286160452 conn(0x7fe4600b50f0 msgr2=0x7fe46015cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 shutdown_connections 2024-03-08T08:27:51.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.590+0000 7fe464ba1640 1 -- 172.21.15.105:0/1286160452 wait complete. 2024-03-08T08:27:51.602 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-03-08T08:27:51.098285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218996,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755724,"statfs":{"total":773094113280,"available":772869861376,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.434491"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285490+0000","last_change":"2024-03-08T08:27:39.669170+0000","last_active":"2024-03-08T08:27:42.285490+0000","last_peered":"2024-03-08T08:27:42.285490+0000","last_clean":"2024-03-08T08:27:42.285490+0000","last_became_active":"2024-03-08T08:27:39.668776+0000","last_became_peered":"2024-03-08T08:27:39.668776+0000","last_unstale":"2024-03-08T08:27:42.285490+0000","last_undegraded":"2024-03-08T08:27:42.285490+0000","last_fullsized":"2024-03-08T08:27:42.285490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:58.646243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665542+0000","last_change":"2024-03-08T08:27:39.669937+0000","last_active":"2024-03-08T08:27:40.665542+0000","last_peered":"2024-03-08T08:27:40.665542+0000","last_clean":"2024-03-08T08:27:40.665542+0000","last_became_active":"2024-03-08T08:27:39.669643+0000","last_became_peered":"2024-03-08T08:27:39.669643+0000","last_unstale":"2024-03-08T08:27:40.665542+0000","last_undegraded":"2024-03-08T08:27:40.665542+0000","last_fullsized":"2024-03-08T08:27:40.665542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229937+0000","last_change":"2024-03-08T08:27:39.669698+0000","last_active":"2024-03-08T08:27:43.229937+0000","last_peered":"2024-03-08T08:27:43.229937+0000","last_clean":"2024-03-08T08:27:43.229937+0000","last_became_active":"2024-03-08T08:27:39.669412+0000","last_became_peered":"2024-03-08T08:27:39.669412+0000","last_unstale":"2024-03-08T08:27:43.229937+0000","last_undegraded":"2024-03-08T08:27:43.229937+0000","last_fullsized":"2024-03-08T08:27:43.229937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229311+0000","last_change":"2024-03-08T08:27:39.674216+0000","last_active":"2024-03-08T08:27:43.229311+0000","last_peered":"2024-03-08T08:27:43.229311+0000","last_clean":"2024-03-08T08:27:43.229311+0000","last_became_active":"2024-03-08T08:27:39.673664+0000","last_became_peered":"2024-03-08T08:27:39.673664+0000","last_unstale":"2024-03-08T08:27:43.229311+0000","last_undegraded":"2024-03-08T08:27:43.229311+0000","last_fullsized":"2024-03-08T08:27:43.229311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285273+0000","last_change":"2024-03-08T08:27:39.671956+0000","last_active":"2024-03-08T08:27:42.285273+0000","last_peered":"2024-03-08T08:27:42.285273+0000","last_clean":"2024-03-08T08:27:42.285273+0000","last_became_active":"2024-03-08T08:27:39.671671+0000","last_became_peered":"2024-03-08T08:27:39.671671+0000","last_unstale":"2024-03-08T08:27:42.285273+0000","last_undegraded":"2024-03-08T08:27:42.285273+0000","last_fullsized":"2024-03-08T08:27:42.285273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229106+0000","last_change":"2024-03-08T08:27:39.681569+0000","last_active":"2024-03-08T08:27:43.229106+0000","last_peered":"2024-03-08T08:27:43.229106+0000","last_clean":"2024-03-08T08:27:43.229106+0000","last_became_active":"2024-03-08T08:27:39.681306+0000","last_became_peered":"2024-03-08T08:27:39.681306+0000","last_unstale":"2024-03-08T08:27:43.229106+0000","last_undegraded":"2024-03-08T08:27:43.229106+0000","last_fullsized":"2024-03-08T08:27:43.229106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285103+0000","last_change":"2024-03-08T08:27:39.669471+0000","last_active":"2024-03-08T08:27:42.285103+0000","last_peered":"2024-03-08T08:27:42.285103+0000","last_clean":"2024-03-08T08:27:42.285103+0000","last_became_active":"2024-03-08T08:27:39.669144+0000","last_became_peered":"2024-03-08T08:27:39.669144+0000","last_unstale":"2024-03-08T08:27:42.285103+0000","last_undegraded":"2024-03-08T08:27:42.285103+0000","last_fullsized":"2024-03-08T08:27:42.285103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228726+0000","last_change":"2024-03-08T08:27:39.680587+0000","last_active":"2024-03-08T08:27:43.228726+0000","last_peered":"2024-03-08T08:27:43.228726+0000","last_clean":"2024-03-08T08:27:43.228726+0000","last_became_active":"2024-03-08T08:27:39.680354+0000","last_became_peered":"2024-03-08T08:27:39.680354+0000","last_unstale":"2024-03-08T08:27:43.228726+0000","last_undegraded":"2024-03-08T08:27:43.228726+0000","last_fullsized":"2024-03-08T08:27:43.228726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285685+0000","last_change":"2024-03-08T08:27:39.682744+0000","last_active":"2024-03-08T08:27:42.285685+0000","last_peered":"2024-03-08T08:27:42.285685+0000","last_clean":"2024-03-08T08:27:42.285685+0000","last_became_active":"2024-03-08T08:27:39.682486+0000","last_became_peered":"2024-03-08T08:27:39.682486+0000","last_unstale":"2024-03-08T08:27:42.285685+0000","last_undegraded":"2024-03-08T08:27:42.285685+0000","last_fullsized":"2024-03-08T08:27:42.285685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:01:26.479387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665493+0000","last_change":"2024-03-08T08:27:39.681720+0000","last_active":"2024-03-08T08:27:40.665493+0000","last_peered":"2024-03-08T08:27:40.665493+0000","last_clean":"2024-03-08T08:27:40.665493+0000","last_became_active":"2024-03-08T08:27:39.681473+0000","last_became_peered":"2024-03-08T08:27:39.681473+0000","last_unstale":"2024-03-08T08:27:40.665493+0000","last_undegraded":"2024-03-08T08:27:40.665493+0000","last_fullsized":"2024-03-08T08:27:40.665493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.681092+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.680824+0000","last_became_peered":"2024-03-08T08:27:39.680824+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665505+0000","last_change":"2024-03-08T08:27:39.674803+0000","last_active":"2024-03-08T08:27:40.665505+0000","last_peered":"2024-03-08T08:27:40.665505+0000","last_clean":"2024-03-08T08:27:40.665505+0000","last_became_active":"2024-03-08T08:27:39.674320+0000","last_became_peered":"2024-03-08T08:27:39.674320+0000","last_unstale":"2024-03-08T08:27:40.665505+0000","last_undegraded":"2024-03-08T08:27:40.665505+0000","last_fullsized":"2024-03-08T08:27:40.665505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228821+0000","last_change":"2024-03-08T08:27:39.665926+0000","last_active":"2024-03-08T08:27:43.228821+0000","last_peered":"2024-03-08T08:27:43.228821+0000","last_clean":"2024-03-08T08:27:43.228821+0000","last_became_active":"2024-03-08T08:27:39.665607+0000","last_became_peered":"2024-03-08T08:27:39.665607+0000","last_unstale":"2024-03-08T08:27:43.228821+0000","last_undegraded":"2024-03-08T08:27:43.228821+0000","last_fullsized":"2024-03-08T08:27:43.228821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229697+0000","last_change":"2024-03-08T08:27:39.666842+0000","last_active":"2024-03-08T08:27:43.229697+0000","last_peered":"2024-03-08T08:27:43.229697+0000","last_clean":"2024-03-08T08:27:43.229697+0000","last_became_active":"2024-03-08T08:27:39.666519+0000","last_became_peered":"2024-03-08T08:27:39.666519+0000","last_unstale":"2024-03-08T08:27:43.229697+0000","last_undegraded":"2024-03-08T08:27:43.229697+0000","last_fullsized":"2024-03-08T08:27:43.229697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:51:51.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665072+0000","last_change":"2024-03-08T08:27:39.672195+0000","last_active":"2024-03-08T08:27:40.665072+0000","last_peered":"2024-03-08T08:27:40.665072+0000","last_clean":"2024-03-08T08:27:40.665072+0000","last_became_active":"2024-03-08T08:27:39.671927+0000","last_became_peered":"2024-03-08T08:27:39.671927+0000","last_unstale":"2024-03-08T08:27:40.665072+0000","last_undegraded":"2024-03-08T08:27:40.665072+0000","last_fullsized":"2024-03-08T08:27:40.665072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229048+0000","last_change":"2024-03-08T08:27:39.677932+0000","last_active":"2024-03-08T08:27:43.229048+0000","last_peered":"2024-03-08T08:27:43.229048+0000","last_clean":"2024-03-08T08:27:43.229048+0000","last_became_active":"2024-03-08T08:27:39.674418+0000","last_became_peered":"2024-03-08T08:27:39.674418+0000","last_unstale":"2024-03-08T08:27:43.229048+0000","last_undegraded":"2024-03-08T08:27:43.229048+0000","last_fullsized":"2024-03-08T08:27:43.229048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229734+0000","last_change":"2024-03-08T08:27:39.680336+0000","last_active":"2024-03-08T08:27:43.229734+0000","last_peered":"2024-03-08T08:27:43.229734+0000","last_clean":"2024-03-08T08:27:43.229734+0000","last_became_active":"2024-03-08T08:27:39.680072+0000","last_became_peered":"2024-03-08T08:27:39.680072+0000","last_unstale":"2024-03-08T08:27:43.229734+0000","last_undegraded":"2024-03-08T08:27:43.229734+0000","last_fullsized":"2024-03-08T08:27:43.229734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:05.572283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228863+0000","last_change":"2024-03-08T08:27:39.663330+0000","last_active":"2024-03-08T08:27:43.228863+0000","last_peered":"2024-03-08T08:27:43.228863+0000","last_clean":"2024-03-08T08:27:43.228863+0000","last_became_active":"2024-03-08T08:27:39.663063+0000","last_became_peered":"2024-03-08T08:27:39.663063+0000","last_unstale":"2024-03-08T08:27:43.228863+0000","last_undegraded":"2024-03-08T08:27:43.228863+0000","last_fullsized":"2024-03-08T08:27:43.228863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:06.280555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285630+0000","last_change":"2024-03-08T08:27:39.673627+0000","last_active":"2024-03-08T08:27:42.285630+0000","last_peered":"2024-03-08T08:27:42.285630+0000","last_clean":"2024-03-08T08:27:42.285630+0000","last_became_active":"2024-03-08T08:27:39.673186+0000","last_became_peered":"2024-03-08T08:27:39.673186+0000","last_unstale":"2024-03-08T08:27:42.285630+0000","last_undegraded":"2024-03-08T08:27:42.285630+0000","last_fullsized":"2024-03-08T08:27:42.285630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:05.572283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664828+0000","last_change":"2024-03-08T08:27:39.672747+0000","last_active":"2024-03-08T08:27:40.664828+0000","last_peered":"2024-03-08T08:27:40.664828+0000","last_clean":"2024-03-08T08:27:40.664828+0000","last_became_active":"2024-03-08T08:27:39.672478+0000","last_became_peered":"2024-03-08T08:27:39.672478+0000","last_unstale":"2024-03-08T08:27:40.664828+0000","last_undegraded":"2024-03-08T08:27:40.664828+0000","last_fullsized":"2024-03-08T08:27:40.664828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229825+0000","last_change":"2024-03-08T08:27:39.671025+0000","last_active":"2024-03-08T08:27:43.229825+0000","last_peered":"2024-03-08T08:27:43.229825+0000","last_clean":"2024-03-08T08:27:43.229825+0000","last_became_active":"2024-03-08T08:27:39.670767+0000","last_became_peered":"2024-03-08T08:27:39.670767+0000","last_unstale":"2024-03-08T08:27:43.229825+0000","last_undegraded":"2024-03-08T08:27:43.229825+0000","last_fullsized":"2024-03-08T08:27:43.229825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:41.578653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229139+0000","last_change":"2024-03-08T08:27:39.680602+0000","last_active":"2024-03-08T08:27:43.229139+0000","last_peered":"2024-03-08T08:27:43.229139+0000","last_clean":"2024-03-08T08:27:43.229139+0000","last_became_active":"2024-03-08T08:27:39.679691+0000","last_became_peered":"2024-03-08T08:27:39.679691+0000","last_unstale":"2024-03-08T08:27:43.229139+0000","last_undegraded":"2024-03-08T08:27:43.229139+0000","last_fullsized":"2024-03-08T08:27:43.229139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229185+0000","last_change":"2024-03-08T08:27:39.673063+0000","last_active":"2024-03-08T08:27:43.229185+0000","last_peered":"2024-03-08T08:27:43.229185+0000","last_clean":"2024-03-08T08:27:43.229185+0000","last_became_active":"2024-03-08T08:27:39.672801+0000","last_became_peered":"2024-03-08T08:27:39.672801+0000","last_unstale":"2024-03-08T08:27:43.229185+0000","last_undegraded":"2024-03-08T08:27:43.229185+0000","last_fullsized":"2024-03-08T08:27:43.229185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229108+0000","last_change":"2024-03-08T08:27:39.680324+0000","last_active":"2024-03-08T08:27:43.229108+0000","last_peered":"2024-03-08T08:27:43.229108+0000","last_clean":"2024-03-08T08:27:43.229108+0000","last_became_active":"2024-03-08T08:27:39.679988+0000","last_became_peered":"2024-03-08T08:27:39.679988+0000","last_unstale":"2024-03-08T08:27:43.229108+0000","last_undegraded":"2024-03-08T08:27:43.229108+0000","last_fullsized":"2024-03-08T08:27:43.229108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228956+0000","last_change":"2024-03-08T08:27:39.669492+0000","last_active":"2024-03-08T08:27:43.228956+0000","last_peered":"2024-03-08T08:27:43.228956+0000","last_clean":"2024-03-08T08:27:43.228956+0000","last_became_active":"2024-03-08T08:27:39.669223+0000","last_became_peered":"2024-03-08T08:27:39.669223+0000","last_unstale":"2024-03-08T08:27:43.228956+0000","last_undegraded":"2024-03-08T08:27:43.228956+0000","last_fullsized":"2024-03-08T08:27:43.228956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665530+0000","last_change":"2024-03-08T08:27:39.675074+0000","last_active":"2024-03-08T08:27:40.665530+0000","last_peered":"2024-03-08T08:27:40.665530+0000","last_clean":"2024-03-08T08:27:40.665530+0000","last_became_active":"2024-03-08T08:27:39.674811+0000","last_became_peered":"2024-03-08T08:27:39.674811+0000","last_unstale":"2024-03-08T08:27:40.665530+0000","last_undegraded":"2024-03-08T08:27:40.665530+0000","last_fullsized":"2024-03-08T08:27:40.665530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665374+0000","last_change":"2024-03-08T08:27:39.666768+0000","last_active":"2024-03-08T08:27:40.665374+0000","last_peered":"2024-03-08T08:27:40.665374+0000","last_clean":"2024-03-08T08:27:40.665374+0000","last_became_active":"2024-03-08T08:27:39.666516+0000","last_became_peered":"2024-03-08T08:27:39.666516+0000","last_unstale":"2024-03-08T08:27:40.665374+0000","last_undegraded":"2024-03-08T08:27:40.665374+0000","last_fullsized":"2024-03-08T08:27:40.665374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229242+0000","last_change":"2024-03-08T08:27:39.672450+0000","last_active":"2024-03-08T08:27:43.229242+0000","last_peered":"2024-03-08T08:27:43.229242+0000","last_clean":"2024-03-08T08:27:43.229242+0000","last_became_active":"2024-03-08T08:27:39.671942+0000","last_became_peered":"2024-03-08T08:27:39.671942+0000","last_unstale":"2024-03-08T08:27:43.229242+0000","last_undegraded":"2024-03-08T08:27:43.229242+0000","last_fullsized":"2024-03-08T08:27:43.229242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229974+0000","last_change":"2024-03-08T08:27:39.668257+0000","last_active":"2024-03-08T08:27:43.229974+0000","last_peered":"2024-03-08T08:27:43.229974+0000","last_clean":"2024-03-08T08:27:43.229974+0000","last_became_active":"2024-03-08T08:27:39.667225+0000","last_became_peered":"2024-03-08T08:27:39.667225+0000","last_unstale":"2024-03-08T08:27:43.229974+0000","last_undegraded":"2024-03-08T08:27:43.229974+0000","last_fullsized":"2024-03-08T08:27:43.229974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229321+0000","last_change":"2024-03-08T08:27:39.680846+0000","last_active":"2024-03-08T08:27:43.229321+0000","last_peered":"2024-03-08T08:27:43.229321+0000","last_clean":"2024-03-08T08:27:43.229321+0000","last_became_active":"2024-03-08T08:27:39.680341+0000","last_became_peered":"2024-03-08T08:27:39.680341+0000","last_unstale":"2024-03-08T08:27:43.229321+0000","last_undegraded":"2024-03-08T08:27:43.229321+0000","last_fullsized":"2024-03-08T08:27:43.229321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229315+0000","last_change":"2024-03-08T08:27:39.679406+0000","last_active":"2024-03-08T08:27:43.229315+0000","last_peered":"2024-03-08T08:27:43.229315+0000","last_clean":"2024-03-08T08:27:43.229315+0000","last_became_active":"2024-03-08T08:27:39.679148+0000","last_became_peered":"2024-03-08T08:27:39.679148+0000","last_unstale":"2024-03-08T08:27:43.229315+0000","last_undegraded":"2024-03-08T08:27:43.229315+0000","last_fullsized":"2024-03-08T08:27:43.229315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:02.805258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664398+0000","last_change":"2024-03-08T08:27:39.670032+0000","last_active":"2024-03-08T08:27:40.664398+0000","last_peered":"2024-03-08T08:27:40.664398+0000","last_clean":"2024-03-08T08:27:40.664398+0000","last_became_active":"2024-03-08T08:27:39.669704+0000","last_became_peered":"2024-03-08T08:27:39.669704+0000","last_unstale":"2024-03-08T08:27:40.664398+0000","last_undegraded":"2024-03-08T08:27:40.664398+0000","last_fullsized":"2024-03-08T08:27:40.664398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:12:44.918819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665149+0000","last_change":"2024-03-08T08:27:39.671209+0000","last_active":"2024-03-08T08:27:40.665149+0000","last_peered":"2024-03-08T08:27:40.665149+0000","last_clean":"2024-03-08T08:27:40.665149+0000","last_became_active":"2024-03-08T08:27:39.670900+0000","last_became_peered":"2024-03-08T08:27:39.670900+0000","last_unstale":"2024-03-08T08:27:40.665149+0000","last_undegraded":"2024-03-08T08:27:40.665149+0000","last_fullsized":"2024-03-08T08:27:40.665149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:46.369514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229061+0000","last_change":"2024-03-08T08:27:38.661099+0000","last_active":"2024-03-08T08:27:43.229061+0000","last_peered":"2024-03-08T08:27:43.229061+0000","last_clean":"2024-03-08T08:27:43.229061+0000","last_became_active":"2024-03-08T08:27:38.660829+0000","last_became_peered":"2024-03-08T08:27:38.660829+0000","last_unstale":"2024-03-08T08:27:43.229061+0000","last_undegraded":"2024-03-08T08:27:43.229061+0000","last_fullsized":"2024-03-08T08:27:43.229061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230116+0000","last_change":"2024-03-08T08:27:39.671575+0000","last_active":"2024-03-08T08:27:43.230116+0000","last_peered":"2024-03-08T08:27:43.230116+0000","last_clean":"2024-03-08T08:27:43.230116+0000","last_became_active":"2024-03-08T08:27:39.671311+0000","last_became_peered":"2024-03-08T08:27:39.671311+0000","last_unstale":"2024-03-08T08:27:43.230116+0000","last_undegraded":"2024-03-08T08:27:43.230116+0000","last_fullsized":"2024-03-08T08:27:43.230116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664688+0000","last_change":"2024-03-08T08:27:38.651812+0000","last_active":"2024-03-08T08:27:40.664688+0000","last_peered":"2024-03-08T08:27:40.664688+0000","last_clean":"2024-03-08T08:27:40.664688+0000","last_became_active":"2024-03-08T08:27:38.651040+0000","last_became_peered":"2024-03-08T08:27:38.651040+0000","last_unstale":"2024-03-08T08:27:40.664688+0000","last_undegraded":"2024-03-08T08:27:40.664688+0000","last_fullsized":"2024-03-08T08:27:40.664688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229327+0000","last_change":"2024-03-08T08:27:38.663033+0000","last_active":"2024-03-08T08:27:43.229327+0000","last_peered":"2024-03-08T08:27:43.229327+0000","last_clean":"2024-03-08T08:27:43.229327+0000","last_became_active":"2024-03-08T08:27:38.662753+0000","last_became_peered":"2024-03-08T08:27:38.662753+0000","last_unstale":"2024-03-08T08:27:43.229327+0000","last_undegraded":"2024-03-08T08:27:43.229327+0000","last_fullsized":"2024-03-08T08:27:43.229327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:13:30.999947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228732+0000","last_change":"2024-03-08T08:27:39.670839+0000","last_active":"2024-03-08T08:27:43.228732+0000","last_peered":"2024-03-08T08:27:43.228732+0000","last_clean":"2024-03-08T08:27:43.228732+0000","last_became_active":"2024-03-08T08:27:39.670559+0000","last_became_peered":"2024-03-08T08:27:39.670559+0000","last_unstale":"2024-03-08T08:27:43.228732+0000","last_undegraded":"2024-03-08T08:27:43.228732+0000","last_fullsized":"2024-03-08T08:27:43.228732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229150+0000","last_change":"2024-03-08T08:27:38.656237+0000","last_active":"2024-03-08T08:27:43.229150+0000","last_peered":"2024-03-08T08:27:43.229150+0000","last_clean":"2024-03-08T08:27:43.229150+0000","last_became_active":"2024-03-08T08:27:38.655944+0000","last_became_peered":"2024-03-08T08:27:38.655944+0000","last_unstale":"2024-03-08T08:27:43.229150+0000","last_undegraded":"2024-03-08T08:27:43.229150+0000","last_fullsized":"2024-03-08T08:27:43.229150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.667881+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.666906+0000","last_became_peered":"2024-03-08T08:27:39.666906+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665029+0000","last_change":"2024-03-08T08:27:38.660261+0000","last_active":"2024-03-08T08:27:40.665029+0000","last_peered":"2024-03-08T08:27:40.665029+0000","last_clean":"2024-03-08T08:27:40.665029+0000","last_became_active":"2024-03-08T08:27:38.659979+0000","last_became_peered":"2024-03-08T08:27:38.659979+0000","last_unstale":"2024-03-08T08:27:40.665029+0000","last_undegraded":"2024-03-08T08:27:40.665029+0000","last_fullsized":"2024-03-08T08:27:40.665029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229622+0000","last_change":"2024-03-08T08:27:39.679621+0000","last_active":"2024-03-08T08:27:43.229622+0000","last_peered":"2024-03-08T08:27:43.229622+0000","last_clean":"2024-03-08T08:27:43.229622+0000","last_became_active":"2024-03-08T08:27:39.679327+0000","last_became_peered":"2024-03-08T08:27:39.679327+0000","last_unstale":"2024-03-08T08:27:43.229622+0000","last_undegraded":"2024-03-08T08:27:43.229622+0000","last_fullsized":"2024-03-08T08:27:43.229622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:07.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229731+0000","last_change":"2024-03-08T08:27:38.662435+0000","last_active":"2024-03-08T08:27:43.229731+0000","last_peered":"2024-03-08T08:27:43.229731+0000","last_clean":"2024-03-08T08:27:43.229731+0000","last_became_active":"2024-03-08T08:27:38.662189+0000","last_became_peered":"2024-03-08T08:27:38.662189+0000","last_unstale":"2024-03-08T08:27:43.229731+0000","last_undegraded":"2024-03-08T08:27:43.229731+0000","last_fullsized":"2024-03-08T08:27:43.229731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230094+0000","last_change":"2024-03-08T08:27:39.680354+0000","last_active":"2024-03-08T08:27:43.230094+0000","last_peered":"2024-03-08T08:27:43.230094+0000","last_clean":"2024-03-08T08:27:43.230094+0000","last_became_active":"2024-03-08T08:27:39.680068+0000","last_became_peered":"2024-03-08T08:27:39.680068+0000","last_unstale":"2024-03-08T08:27:43.230094+0000","last_undegraded":"2024-03-08T08:27:43.230094+0000","last_fullsized":"2024-03-08T08:27:43.230094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:19:46.364928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665344+0000","last_change":"2024-03-08T08:27:38.662811+0000","last_active":"2024-03-08T08:27:40.665344+0000","last_peered":"2024-03-08T08:27:40.665344+0000","last_clean":"2024-03-08T08:27:40.665344+0000","last_became_active":"2024-03-08T08:27:38.662484+0000","last_became_peered":"2024-03-08T08:27:38.662484+0000","last_unstale":"2024-03-08T08:27:40.665344+0000","last_undegraded":"2024-03-08T08:27:40.665344+0000","last_fullsized":"2024-03-08T08:27:40.665344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665494+0000","last_change":"2024-03-08T08:27:39.681653+0000","last_active":"2024-03-08T08:27:40.665494+0000","last_peered":"2024-03-08T08:27:40.665494+0000","last_clean":"2024-03-08T08:27:40.665494+0000","last_became_active":"2024-03-08T08:27:39.681368+0000","last_became_peered":"2024-03-08T08:27:39.681368+0000","last_unstale":"2024-03-08T08:27:40.665494+0000","last_undegraded":"2024-03-08T08:27:40.665494+0000","last_fullsized":"2024-03-08T08:27:40.665494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:48.361535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664972+0000","last_change":"2024-03-08T08:27:38.655367+0000","last_active":"2024-03-08T08:27:40.664972+0000","last_peered":"2024-03-08T08:27:40.664972+0000","last_clean":"2024-03-08T08:27:40.664972+0000","last_became_active":"2024-03-08T08:27:38.655051+0000","last_became_peered":"2024-03-08T08:27:38.655051+0000","last_unstale":"2024-03-08T08:27:40.664972+0000","last_undegraded":"2024-03-08T08:27:40.664972+0000","last_fullsized":"2024-03-08T08:27:40.664972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285287+0000","last_change":"2024-03-08T08:27:39.672887+0000","last_active":"2024-03-08T08:27:42.285287+0000","last_peered":"2024-03-08T08:27:42.285287+0000","last_clean":"2024-03-08T08:27:42.285287+0000","last_became_active":"2024-03-08T08:27:39.672009+0000","last_became_peered":"2024-03-08T08:27:39.672009+0000","last_unstale":"2024-03-08T08:27:42.285287+0000","last_undegraded":"2024-03-08T08:27:42.285287+0000","last_fullsized":"2024-03-08T08:27:42.285287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:36:52.998639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230150+0000","last_change":"2024-03-08T08:27:38.650529+0000","last_active":"2024-03-08T08:27:43.230150+0000","last_peered":"2024-03-08T08:27:43.230150+0000","last_clean":"2024-03-08T08:27:43.230150+0000","last_became_active":"2024-03-08T08:27:38.650183+0000","last_became_peered":"2024-03-08T08:27:38.650183+0000","last_unstale":"2024-03-08T08:27:43.230150+0000","last_undegraded":"2024-03-08T08:27:43.230150+0000","last_fullsized":"2024-03-08T08:27:43.230150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:18:22.093931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229702+0000","last_change":"2024-03-08T08:27:39.670409+0000","last_active":"2024-03-08T08:27:43.229702+0000","last_peered":"2024-03-08T08:27:43.229702+0000","last_clean":"2024-03-08T08:27:43.229702+0000","last_became_active":"2024-03-08T08:27:39.670105+0000","last_became_peered":"2024-03-08T08:27:39.670105+0000","last_unstale":"2024-03-08T08:27:43.229702+0000","last_undegraded":"2024-03-08T08:27:43.229702+0000","last_fullsized":"2024-03-08T08:27:43.229702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:31:15.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229241+0000","last_change":"2024-03-08T08:27:38.660419+0000","last_active":"2024-03-08T08:27:43.229241+0000","last_peered":"2024-03-08T08:27:43.229241+0000","last_clean":"2024-03-08T08:27:43.229241+0000","last_became_active":"2024-03-08T08:27:38.660225+0000","last_became_peered":"2024-03-08T08:27:38.660225+0000","last_unstale":"2024-03-08T08:27:43.229241+0000","last_undegraded":"2024-03-08T08:27:43.229241+0000","last_fullsized":"2024-03-08T08:27:43.229241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285354+0000","last_change":"2024-03-08T08:27:39.675352+0000","last_active":"2024-03-08T08:27:42.285354+0000","last_peered":"2024-03-08T08:27:42.285354+0000","last_clean":"2024-03-08T08:27:42.285354+0000","last_became_active":"2024-03-08T08:27:39.674881+0000","last_became_peered":"2024-03-08T08:27:39.674881+0000","last_unstale":"2024-03-08T08:27:42.285354+0000","last_undegraded":"2024-03-08T08:27:42.285354+0000","last_fullsized":"2024-03-08T08:27:42.285354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664467+0000","last_change":"2024-03-08T08:27:38.656182+0000","last_active":"2024-03-08T08:27:40.664467+0000","last_peered":"2024-03-08T08:27:40.664467+0000","last_clean":"2024-03-08T08:27:40.664467+0000","last_became_active":"2024-03-08T08:27:38.655315+0000","last_became_peered":"2024-03-08T08:27:38.655315+0000","last_unstale":"2024-03-08T08:27:40.664467+0000","last_undegraded":"2024-03-08T08:27:40.664467+0000","last_fullsized":"2024-03-08T08:27:40.664467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664532+0000","last_change":"2024-03-08T08:27:39.666898+0000","last_active":"2024-03-08T08:27:40.664532+0000","last_peered":"2024-03-08T08:27:40.664532+0000","last_clean":"2024-03-08T08:27:40.664532+0000","last_became_active":"2024-03-08T08:27:39.666563+0000","last_became_peered":"2024-03-08T08:27:39.666563+0000","last_unstale":"2024-03-08T08:27:40.664532+0000","last_undegraded":"2024-03-08T08:27:40.664532+0000","last_fullsized":"2024-03-08T08:27:40.664532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229594+0000","last_change":"2024-03-08T08:27:38.656580+0000","last_active":"2024-03-08T08:27:43.229594+0000","last_peered":"2024-03-08T08:27:43.229594+0000","last_clean":"2024-03-08T08:27:43.229594+0000","last_became_active":"2024-03-08T08:27:38.656233+0000","last_became_peered":"2024-03-08T08:27:38.656233+0000","last_unstale":"2024-03-08T08:27:43.229594+0000","last_undegraded":"2024-03-08T08:27:43.229594+0000","last_fullsized":"2024-03-08T08:27:43.229594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285501+0000","last_change":"2024-03-08T08:27:39.677499+0000","last_active":"2024-03-08T08:27:42.285501+0000","last_peered":"2024-03-08T08:27:42.285501+0000","last_clean":"2024-03-08T08:27:42.285501+0000","last_became_active":"2024-03-08T08:27:39.677213+0000","last_became_peered":"2024-03-08T08:27:39.677213+0000","last_unstale":"2024-03-08T08:27:42.285501+0000","last_undegraded":"2024-03-08T08:27:42.285501+0000","last_fullsized":"2024-03-08T08:27:42.285501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:00.627737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665133+0000","last_change":"2024-03-08T08:27:38.653100+0000","last_active":"2024-03-08T08:27:40.665133+0000","last_peered":"2024-03-08T08:27:40.665133+0000","last_clean":"2024-03-08T08:27:40.665133+0000","last_became_active":"2024-03-08T08:27:38.652837+0000","last_became_peered":"2024-03-08T08:27:38.652837+0000","last_unstale":"2024-03-08T08:27:40.665133+0000","last_undegraded":"2024-03-08T08:27:40.665133+0000","last_fullsized":"2024-03-08T08:27:40.665133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285597+0000","last_change":"2024-03-08T08:27:39.673698+0000","last_active":"2024-03-08T08:27:42.285597+0000","last_peered":"2024-03-08T08:27:42.285597+0000","last_clean":"2024-03-08T08:27:42.285597+0000","last_became_active":"2024-03-08T08:27:39.673424+0000","last_became_peered":"2024-03-08T08:27:39.673424+0000","last_unstale":"2024-03-08T08:27:42.285597+0000","last_undegraded":"2024-03-08T08:27:42.285597+0000","last_fullsized":"2024-03-08T08:27:42.285597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:12:43.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228967+0000","last_change":"2024-03-08T08:27:38.660277+0000","last_active":"2024-03-08T08:27:43.228967+0000","last_peered":"2024-03-08T08:27:43.228967+0000","last_clean":"2024-03-08T08:27:43.228967+0000","last_became_active":"2024-03-08T08:27:38.659873+0000","last_became_peered":"2024-03-08T08:27:38.659873+0000","last_unstale":"2024-03-08T08:27:43.228967+0000","last_undegraded":"2024-03-08T08:27:43.228967+0000","last_fullsized":"2024-03-08T08:27:43.228967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285694+0000","last_change":"2024-03-08T08:27:39.674403+0000","last_active":"2024-03-08T08:27:42.285694+0000","last_peered":"2024-03-08T08:27:42.285694+0000","last_clean":"2024-03-08T08:27:42.285694+0000","last_became_active":"2024-03-08T08:27:39.674152+0000","last_became_peered":"2024-03-08T08:27:39.674152+0000","last_unstale":"2024-03-08T08:27:42.285694+0000","last_undegraded":"2024-03-08T08:27:42.285694+0000","last_fullsized":"2024-03-08T08:27:42.285694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:58:36.475015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665435+0000","last_change":"2024-03-08T08:27:38.655005+0000","last_active":"2024-03-08T08:27:40.665435+0000","last_peered":"2024-03-08T08:27:40.665435+0000","last_clean":"2024-03-08T08:27:40.665435+0000","last_became_active":"2024-03-08T08:27:38.654683+0000","last_became_peered":"2024-03-08T08:27:38.654683+0000","last_unstale":"2024-03-08T08:27:40.665435+0000","last_undegraded":"2024-03-08T08:27:40.665435+0000","last_fullsized":"2024-03-08T08:27:40.665435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229535+0000","last_change":"2024-03-08T08:27:39.673959+0000","last_active":"2024-03-08T08:27:43.229535+0000","last_peered":"2024-03-08T08:27:43.229535+0000","last_clean":"2024-03-08T08:27:43.229535+0000","last_became_active":"2024-03-08T08:27:39.673369+0000","last_became_peered":"2024-03-08T08:27:39.673369+0000","last_unstale":"2024-03-08T08:27:43.229535+0000","last_undegraded":"2024-03-08T08:27:43.229535+0000","last_fullsized":"2024-03-08T08:27:43.229535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:00.627737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229488+0000","last_change":"2024-03-08T08:27:38.661161+0000","last_active":"2024-03-08T08:27:43.229488+0000","last_peered":"2024-03-08T08:27:43.229488+0000","last_clean":"2024-03-08T08:27:43.229488+0000","last_became_active":"2024-03-08T08:27:38.660876+0000","last_became_peered":"2024-03-08T08:27:38.660876+0000","last_unstale":"2024-03-08T08:27:43.229488+0000","last_undegraded":"2024-03-08T08:27:43.229488+0000","last_fullsized":"2024-03-08T08:27:43.229488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285472+0000","last_change":"2024-03-08T08:27:39.682454+0000","last_active":"2024-03-08T08:27:42.285472+0000","last_peered":"2024-03-08T08:27:42.285472+0000","last_clean":"2024-03-08T08:27:42.285472+0000","last_became_active":"2024-03-08T08:27:39.682178+0000","last_became_peered":"2024-03-08T08:27:39.682178+0000","last_unstale":"2024-03-08T08:27:42.285472+0000","last_undegraded":"2024-03-08T08:27:42.285472+0000","last_fullsized":"2024-03-08T08:27:42.285472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:07.461706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285074+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:27:42.285074+0000","last_peered":"2024-03-08T08:27:42.285074+0000","last_clean":"2024-03-08T08:27:42.285074+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:27:42.285074+0000","last_undegraded":"2024-03-08T08:27:42.285074+0000","last_fullsized":"2024-03-08T08:27:42.285074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229052+0000","last_change":"2024-03-08T08:27:38.659294+0000","last_active":"2024-03-08T08:27:43.229052+0000","last_peered":"2024-03-08T08:27:43.229052+0000","last_clean":"2024-03-08T08:27:43.229052+0000","last_became_active":"2024-03-08T08:27:38.659037+0000","last_became_peered":"2024-03-08T08:27:38.659037+0000","last_unstale":"2024-03-08T08:27:43.229052+0000","last_undegraded":"2024-03-08T08:27:43.229052+0000","last_fullsized":"2024-03-08T08:27:43.229052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:25.078138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228947+0000","last_change":"2024-03-08T08:27:39.681265+0000","last_active":"2024-03-08T08:27:43.228947+0000","last_peered":"2024-03-08T08:27:43.228947+0000","last_clean":"2024-03-08T08:27:43.228947+0000","last_became_active":"2024-03-08T08:27:39.680995+0000","last_became_peered":"2024-03-08T08:27:39.680995+0000","last_unstale":"2024-03-08T08:27:43.228947+0000","last_undegraded":"2024-03-08T08:27:43.228947+0000","last_fullsized":"2024-03-08T08:27:43.228947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:54:41.848622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.284946+0000","last_change":"2024-03-08T08:27:38.663815+0000","last_active":"2024-03-08T08:27:42.284946+0000","last_peered":"2024-03-08T08:27:42.284946+0000","last_clean":"2024-03-08T08:27:42.284946+0000","last_became_active":"2024-03-08T08:27:38.663589+0000","last_became_peered":"2024-03-08T08:27:38.663589+0000","last_unstale":"2024-03-08T08:27:42.284946+0000","last_undegraded":"2024-03-08T08:27:42.284946+0000","last_fullsized":"2024-03-08T08:27:42.284946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229606+0000","last_change":"2024-03-08T08:27:39.673596+0000","last_active":"2024-03-08T08:27:43.229606+0000","last_peered":"2024-03-08T08:27:43.229606+0000","last_clean":"2024-03-08T08:27:43.229606+0000","last_became_active":"2024-03-08T08:27:39.673171+0000","last_became_peered":"2024-03-08T08:27:39.673171+0000","last_unstale":"2024-03-08T08:27:43.229606+0000","last_undegraded":"2024-03-08T08:27:43.229606+0000","last_fullsized":"2024-03-08T08:27:43.229606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:22:43.266495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229195+0000","last_change":"2024-03-08T08:27:38.658281+0000","last_active":"2024-03-08T08:27:43.229195+0000","last_peered":"2024-03-08T08:27:43.229195+0000","last_clean":"2024-03-08T08:27:43.229195+0000","last_became_active":"2024-03-08T08:27:38.657983+0000","last_became_peered":"2024-03-08T08:27:38.657983+0000","last_unstale":"2024-03-08T08:27:43.229195+0000","last_undegraded":"2024-03-08T08:27:43.229195+0000","last_fullsized":"2024-03-08T08:27:43.229195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665068+0000","last_change":"2024-03-08T08:27:39.662315+0000","last_active":"2024-03-08T08:27:40.665068+0000","last_peered":"2024-03-08T08:27:40.665068+0000","last_clean":"2024-03-08T08:27:40.665068+0000","last_became_active":"2024-03-08T08:27:39.661362+0000","last_became_peered":"2024-03-08T08:27:39.661362+0000","last_unstale":"2024-03-08T08:27:40.665068+0000","last_undegraded":"2024-03-08T08:27:40.665068+0000","last_fullsized":"2024-03-08T08:27:40.665068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229349+0000","last_change":"2024-03-08T08:27:38.659980+0000","last_active":"2024-03-08T08:27:43.229349+0000","last_peered":"2024-03-08T08:27:43.229349+0000","last_clean":"2024-03-08T08:27:43.229349+0000","last_became_active":"2024-03-08T08:27:38.659704+0000","last_became_peered":"2024-03-08T08:27:38.659704+0000","last_unstale":"2024-03-08T08:27:43.229349+0000","last_undegraded":"2024-03-08T08:27:43.229349+0000","last_fullsized":"2024-03-08T08:27:43.229349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665238+0000","last_change":"2024-03-08T08:27:39.678619+0000","last_active":"2024-03-08T08:27:40.665238+0000","last_peered":"2024-03-08T08:27:40.665238+0000","last_clean":"2024-03-08T08:27:40.665238+0000","last_became_active":"2024-03-08T08:27:39.678371+0000","last_became_peered":"2024-03-08T08:27:39.678371+0000","last_unstale":"2024-03-08T08:27:40.665238+0000","last_undegraded":"2024-03-08T08:27:40.665238+0000","last_fullsized":"2024-03-08T08:27:40.665238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:13.633091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665304+0000","last_change":"2024-03-08T08:27:38.654132+0000","last_active":"2024-03-08T08:27:40.665304+0000","last_peered":"2024-03-08T08:27:40.665304+0000","last_clean":"2024-03-08T08:27:40.665304+0000","last_became_active":"2024-03-08T08:27:38.653844+0000","last_became_peered":"2024-03-08T08:27:38.653844+0000","last_unstale":"2024-03-08T08:27:40.665304+0000","last_undegraded":"2024-03-08T08:27:40.665304+0000","last_fullsized":"2024-03-08T08:27:40.665304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229651+0000","last_change":"2024-03-08T08:27:39.679299+0000","last_active":"2024-03-08T08:27:43.229651+0000","last_peered":"2024-03-08T08:27:43.229651+0000","last_clean":"2024-03-08T08:27:43.229651+0000","last_became_active":"2024-03-08T08:27:39.679023+0000","last_became_peered":"2024-03-08T08:27:39.679023+0000","last_unstale":"2024-03-08T08:27:43.229651+0000","last_undegraded":"2024-03-08T08:27:43.229651+0000","last_fullsized":"2024-03-08T08:27:43.229651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:12:43.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285620+0000","last_change":"2024-03-08T08:27:38.665656+0000","last_active":"2024-03-08T08:27:42.285620+0000","last_peered":"2024-03-08T08:27:42.285620+0000","last_clean":"2024-03-08T08:27:42.285620+0000","last_became_active":"2024-03-08T08:27:38.665370+0000","last_became_peered":"2024-03-08T08:27:38.665370+0000","last_unstale":"2024-03-08T08:27:42.285620+0000","last_undegraded":"2024-03-08T08:27:42.285620+0000","last_fullsized":"2024-03-08T08:27:42.285620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:52:01.800668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665354+0000","last_change":"2024-03-08T08:27:39.664052+0000","last_active":"2024-03-08T08:27:40.665354+0000","last_peered":"2024-03-08T08:27:40.665354+0000","last_clean":"2024-03-08T08:27:40.665354+0000","last_became_active":"2024-03-08T08:27:39.663794+0000","last_became_peered":"2024-03-08T08:27:39.663794+0000","last_unstale":"2024-03-08T08:27:40.665354+0000","last_undegraded":"2024-03-08T08:27:40.665354+0000","last_fullsized":"2024-03-08T08:27:40.665354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:09:58.884494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:41.113731+0000","last_change":"2024-03-08T08:27:38.658214+0000","last_active":"2024-03-08T08:27:41.113731+0000","last_peered":"2024-03-08T08:27:41.113731+0000","last_clean":"2024-03-08T08:27:41.113731+0000","last_became_active":"2024-03-08T08:27:38.657944+0000","last_became_peered":"2024-03-08T08:27:38.657944+0000","last_unstale":"2024-03-08T08:27:41.113731+0000","last_undegraded":"2024-03-08T08:27:41.113731+0000","last_fullsized":"2024-03-08T08:27:41.113731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229036+0000","last_change":"2024-03-08T08:27:39.671410+0000","last_active":"2024-03-08T08:27:43.229036+0000","last_peered":"2024-03-08T08:27:43.229036+0000","last_clean":"2024-03-08T08:27:43.229036+0000","last_became_active":"2024-03-08T08:27:39.671169+0000","last_became_peered":"2024-03-08T08:27:39.671169+0000","last_unstale":"2024-03-08T08:27:43.229036+0000","last_undegraded":"2024-03-08T08:27:43.229036+0000","last_fullsized":"2024-03-08T08:27:43.229036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:05.230271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665140+0000","last_change":"2024-03-08T08:27:38.657833+0000","last_active":"2024-03-08T08:27:40.665140+0000","last_peered":"2024-03-08T08:27:40.665140+0000","last_clean":"2024-03-08T08:27:40.665140+0000","last_became_active":"2024-03-08T08:27:38.657560+0000","last_became_peered":"2024-03-08T08:27:38.657560+0000","last_unstale":"2024-03-08T08:27:40.665140+0000","last_undegraded":"2024-03-08T08:27:40.665140+0000","last_fullsized":"2024-03-08T08:27:40.665140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:04:45.364923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230104+0000","last_change":"2024-03-08T08:27:39.668821+0000","last_active":"2024-03-08T08:27:43.230104+0000","last_peered":"2024-03-08T08:27:43.230104+0000","last_clean":"2024-03-08T08:27:43.230104+0000","last_became_active":"2024-03-08T08:27:39.668559+0000","last_became_peered":"2024-03-08T08:27:39.668559+0000","last_unstale":"2024-03-08T08:27:43.230104+0000","last_undegraded":"2024-03-08T08:27:43.230104+0000","last_fullsized":"2024-03-08T08:27:43.230104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.284921+0000","last_change":"2024-03-08T08:27:38.665148+0000","last_active":"2024-03-08T08:27:42.284921+0000","last_peered":"2024-03-08T08:27:42.284921+0000","last_clean":"2024-03-08T08:27:42.284921+0000","last_became_active":"2024-03-08T08:27:38.664834+0000","last_became_peered":"2024-03-08T08:27:38.664834+0000","last_unstale":"2024-03-08T08:27:42.284921+0000","last_undegraded":"2024-03-08T08:27:42.284921+0000","last_fullsized":"2024-03-08T08:27:42.284921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664807+0000","last_change":"2024-03-08T08:27:39.667641+0000","last_active":"2024-03-08T08:27:40.664807+0000","last_peered":"2024-03-08T08:27:40.664807+0000","last_clean":"2024-03-08T08:27:40.664807+0000","last_became_active":"2024-03-08T08:27:39.667296+0000","last_became_peered":"2024-03-08T08:27:39.667296+0000","last_unstale":"2024-03-08T08:27:40.664807+0000","last_undegraded":"2024-03-08T08:27:40.664807+0000","last_fullsized":"2024-03-08T08:27:40.664807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:53.145163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665163+0000","last_change":"2024-03-08T08:27:38.660688+0000","last_active":"2024-03-08T08:27:40.665163+0000","last_peered":"2024-03-08T08:27:40.665163+0000","last_clean":"2024-03-08T08:27:40.665163+0000","last_became_active":"2024-03-08T08:27:38.660427+0000","last_became_peered":"2024-03-08T08:27:38.660427+0000","last_unstale":"2024-03-08T08:27:40.665163+0000","last_undegraded":"2024-03-08T08:27:40.665163+0000","last_fullsized":"2024-03-08T08:27:40.665163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228902+0000","last_change":"2024-03-08T08:27:39.665202+0000","last_active":"2024-03-08T08:27:43.228902+0000","last_peered":"2024-03-08T08:27:43.228902+0000","last_clean":"2024-03-08T08:27:43.228902+0000","last_became_active":"2024-03-08T08:27:39.663567+0000","last_became_peered":"2024-03-08T08:27:39.663567+0000","last_unstale":"2024-03-08T08:27:43.228902+0000","last_undegraded":"2024-03-08T08:27:43.228902+0000","last_fullsized":"2024-03-08T08:27:43.228902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:54:33.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.665293+0000","last_change":"2024-03-08T08:27:38.660223+0000","last_active":"2024-03-08T08:27:40.665293+0000","last_peered":"2024-03-08T08:27:40.665293+0000","last_clean":"2024-03-08T08:27:40.665293+0000","last_became_active":"2024-03-08T08:27:38.659939+0000","last_became_peered":"2024-03-08T08:27:38.659939+0000","last_unstale":"2024-03-08T08:27:40.665293+0000","last_undegraded":"2024-03-08T08:27:40.665293+0000","last_fullsized":"2024-03-08T08:27:40.665293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:52.140572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.228969+0000","last_change":"2024-03-08T08:27:39.670191+0000","last_active":"2024-03-08T08:27:43.228969+0000","last_peered":"2024-03-08T08:27:43.228969+0000","last_clean":"2024-03-08T08:27:43.228969+0000","last_became_active":"2024-03-08T08:27:39.669921+0000","last_became_peered":"2024-03-08T08:27:39.669921+0000","last_unstale":"2024-03-08T08:27:43.228969+0000","last_undegraded":"2024-03-08T08:27:43.228969+0000","last_fullsized":"2024-03-08T08:27:43.228969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:12:37.749902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285389+0000","last_change":"2024-03-08T08:27:38.665118+0000","last_active":"2024-03-08T08:27:42.285389+0000","last_peered":"2024-03-08T08:27:42.285389+0000","last_clean":"2024-03-08T08:27:42.285389+0000","last_became_active":"2024-03-08T08:27:38.664812+0000","last_became_peered":"2024-03-08T08:27:38.664812+0000","last_unstale":"2024-03-08T08:27:42.285389+0000","last_undegraded":"2024-03-08T08:27:42.285389+0000","last_fullsized":"2024-03-08T08:27:42.285389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:40:47.356944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:40.664566+0000","last_change":"2024-03-08T08:27:39.672103+0000","last_active":"2024-03-08T08:27:40.664566+0000","last_peered":"2024-03-08T08:27:40.664566+0000","last_clean":"2024-03-08T08:27:40.664566+0000","last_became_active":"2024-03-08T08:27:39.671821+0000","last_became_peered":"2024-03-08T08:27:39.671821+0000","last_unstale":"2024-03-08T08:27:40.664566+0000","last_undegraded":"2024-03-08T08:27:40.664566+0000","last_fullsized":"2024-03-08T08:27:40.664566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:15:37.143799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229542+0000","last_change":"2024-03-08T08:27:39.676821+0000","last_active":"2024-03-08T08:27:43.229542+0000","last_peered":"2024-03-08T08:27:43.229542+0000","last_clean":"2024-03-08T08:27:43.229542+0000","last_became_active":"2024-03-08T08:27:39.674571+0000","last_became_peered":"2024-03-08T08:27:39.674571+0000","last_unstale":"2024-03-08T08:27:43.229542+0000","last_undegraded":"2024-03-08T08:27:43.229542+0000","last_fullsized":"2024-03-08T08:27:43.229542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:58.646243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285171+0000","last_change":"2024-03-08T08:27:38.667618+0000","last_active":"2024-03-08T08:27:42.285171+0000","last_peered":"2024-03-08T08:27:42.285171+0000","last_clean":"2024-03-08T08:27:42.285171+0000","last_became_active":"2024-03-08T08:27:38.667361+0000","last_became_peered":"2024-03-08T08:27:38.667361+0000","last_unstale":"2024-03-08T08:27:42.285171+0000","last_undegraded":"2024-03-08T08:27:42.285171+0000","last_fullsized":"2024-03-08T08:27:42.285171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:57:12.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229446+0000","last_change":"2024-03-08T08:27:39.679501+0000","last_active":"2024-03-08T08:27:43.229446+0000","last_peered":"2024-03-08T08:27:43.229446+0000","last_clean":"2024-03-08T08:27:43.229446+0000","last_became_active":"2024-03-08T08:27:39.679204+0000","last_became_peered":"2024-03-08T08:27:39.679204+0000","last_unstale":"2024-03-08T08:27:43.229446+0000","last_undegraded":"2024-03-08T08:27:43.229446+0000","last_fullsized":"2024-03-08T08:27:43.229446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:07.916902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:42.285442+0000","last_change":"2024-03-08T08:27:38.667616+0000","last_active":"2024-03-08T08:27:42.285442+0000","last_peered":"2024-03-08T08:27:42.285442+0000","last_clean":"2024-03-08T08:27:42.285442+0000","last_became_active":"2024-03-08T08:27:38.667335+0000","last_became_peered":"2024-03-08T08:27:38.667335+0000","last_unstale":"2024-03-08T08:27:42.285442+0000","last_undegraded":"2024-03-08T08:27:42.285442+0000","last_fullsized":"2024-03-08T08:27:42.285442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:27:05.275964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229948+0000","last_change":"2024-03-08T08:27:39.681042+0000","last_active":"2024-03-08T08:27:43.229948+0000","last_peered":"2024-03-08T08:27:43.229948+0000","last_clean":"2024-03-08T08:27:43.229948+0000","last_became_active":"2024-03-08T08:27:39.680791+0000","last_became_peered":"2024-03-08T08:27:39.680791+0000","last_unstale":"2024-03-08T08:27:43.229948+0000","last_undegraded":"2024-03-08T08:27:43.229948+0000","last_fullsized":"2024-03-08T08:27:43.229948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:41:38.361113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229414+0000","last_change":"2024-03-08T08:27:38.659145+0000","last_active":"2024-03-08T08:27:43.229414+0000","last_peered":"2024-03-08T08:27:43.229414+0000","last_clean":"2024-03-08T08:27:43.229414+0000","last_became_active":"2024-03-08T08:27:38.658848+0000","last_became_peered":"2024-03-08T08:27:38.658848+0000","last_unstale":"2024-03-08T08:27:43.229414+0000","last_undegraded":"2024-03-08T08:27:43.229414+0000","last_fullsized":"2024-03-08T08:27:43.229414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:32:15.839644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.229123+0000","last_change":"2024-03-08T08:27:39.668041+0000","last_active":"2024-03-08T08:27:43.229123+0000","last_peered":"2024-03-08T08:27:43.229123+0000","last_clean":"2024-03-08T08:27:43.229123+0000","last_became_active":"2024-03-08T08:27:39.667724+0000","last_became_peered":"2024-03-08T08:27:39.667724+0000","last_unstale":"2024-03-08T08:27:43.229123+0000","last_undegraded":"2024-03-08T08:27:43.229123+0000","last_fullsized":"2024-03-08T08:27:43.229123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:38.637143+0000","last_clean_scrub_stamp":"2024-03-08T08:27:38.637143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:03:34.623348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-08T08:27:43.230114+0000","last_change":"2024-03-08T08:27:38.657913+0000","last_active":"2024-03-08T08:27:43.230114+0000","last_peered":"2024-03-08T08:27:43.230114+0000","last_clean":"2024-03-08T08:27:43.230114+0000","last_became_active":"2024-03-08T08:27:38.657593+0000","last_became_peered":"2024-03-08T08:27:38.657593+0000","last_unstale":"2024-03-08T08:27:43.230114+0000","last_undegraded":"2024-03-08T08:27:43.230114+0000","last_fullsized":"2024-03-08T08:27:43.230114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:37.632553+0000","last_clean_scrub_stamp":"2024-03-08T08:27:37.632553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:21:42.577099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:27:51.606 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-08T08:27:51.607 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-08T08:27:51.607 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-08T08:27:51.607 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-08T08:27:51.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.741+0000 7f238e084640 1 Processor -- start 2024-03-08T08:27:51.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.741+0000 7f238e084640 1 -- start start 2024-03-08T08:27:51.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f238e084640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23880b1550 0x7f23880aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f238e084640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 0x7f23880b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f238e084640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880b1af0 con 0x7f238815dd80 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f238e084640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880b2700 con 0x7f23880b1550 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f238e084640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23880b1dd0 con 0x7f23880b2b90 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23877fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f238815dd80 0x7f23881623c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40382/0 (socket says 172.21.15.105:40382) 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23877fe640 1 -- 172.21.15.105:0/1232728916 learned_addr learned my addr 172.21.15.105:0/1232728916 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f2387fff640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 0x7f23880b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f2386ffd640 1 --2- 172.21.15.105:0/1232728916 >> v2:172.21.15.144:3300/0 conn(0x7f23880b1550 0x7f23880aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548989434 0 0) 0x7f23880b1af0 con 0x7f238815dd80 2024-03-08T08:27:51.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2370003520 con 0x7f238815dd80 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1172774776 0 0) 0x7f2370003520 con 0x7f238815dd80 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 msgr2=0x7f23880b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 0x7f23880b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 >> v2:172.21.15.144:3300/0 conn(0x7f23880b1550 msgr2=0x7f23880aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 --2- 172.21.15.105:0/1232728916 >> v2:172.21.15.144:3300/0 conn(0x7f23880b1550 0x7f23880aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880b20b0 con 0x7f238815dd80 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f2378001670 con 0x7f238815dd80 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 >> v1:172.21.15.105:6789/0 conn(0x7f238815dd80 legacy=0x7f23881623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23700039b0 0x7f2370003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2370004800 0x7f2370004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 0x7f23700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880b2700 con 0x7f23700039b0 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880b1dd0 con 0x7f2370004800 2024-03-08T08:27:51.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237000b9c0 con 0x7f23880b2b90 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.742+0000 7f23877fe640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23700039b0 0x7f2370003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f2386ffd640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2370004800 0x7f2370004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f2387fff640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 0x7f23700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f23877fe640 1 -- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 msgr2=0x7f23700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f23877fe640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 0x7f23700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f23877fe640 1 -- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2370004800 msgr2=0x7f2370004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f23877fe640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2370004800 0x7f2370004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f23877fe640 1 -- 172.21.15.105:0/1232728916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2370003520 con 0x7f23700039b0 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f2387fff640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23880b2b90 0x7f23700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.743+0000 7f23877fe640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23700039b0 0x7f2370003dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f237800c290 tx=0x7f2370005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f513f2da0eda61 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378007d30 con 0x7f23700039b0 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2378030070 con 0x7f23700039b0 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f23867fc640 1 -- 172.21.15.105:0/1232728916 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378005d30 con 0x7f23700039b0 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f238e084640 1 -- 172.21.15.105:0/1232728916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23700039b0 msgr2=0x7f2370003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f238e084640 1 --2- 172.21.15.105:0/1232728916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23700039b0 0x7f2370003dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f237800c290 tx=0x7f2370005830 comp rx=0 tx=0).stop 2024-03-08T08:27:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f2387fff640 1 -- 172.21.15.105:0/1232728916 reap_dead start 2024-03-08T08:27:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f238e084640 1 -- 172.21.15.105:0/1232728916 shutdown_connections 2024-03-08T08:27:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f238e084640 1 -- 172.21.15.105:0/1232728916 >> 172.21.15.105:0/1232728916 conn(0x7f23880b37a0 msgr2=0x7f23880b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f238e084640 1 -- 172.21.15.105:0/1232728916 shutdown_connections 2024-03-08T08:27:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.744+0000 7f238e084640 1 -- 172.21.15.105:0/1232728916 wait complete. 2024-03-08T08:27:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.745+0000 7f238e084640 1 Processor -- start 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.745+0000 7f238e084640 1 -- start start 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f238e084640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23700039b0 0x7f2388146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f238e084640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23881f8090 0x7f238815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f238e084640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23881f7410 0x7f23881f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f238e084640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238815e160 con 0x7f23881f7410 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f238e084640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880aefc0 con 0x7f23881f8090 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f238e084640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238815d4b0 con 0x7f23700039b0 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2387fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23881f7410 0x7f23881f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2387fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23881f7410 0x7f23881f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59742/0 (socket says 172.21.15.105:59742) 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2387fff640 1 -- 172.21.15.105:0/2046264488 learned_addr learned my addr 172.21.15.105:0/2046264488 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f23877fe640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23700039b0 0x7f2388146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2386ffd640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23881f8090 0x7f238815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2387fff640 1 -- 172.21.15.105:0/2046264488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23700039b0 msgr2=0x7f2388146c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2387fff640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23700039b0 0x7f2388146c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2387fff640 1 -- 172.21.15.105:0/2046264488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23881f8090 msgr2=0x7f238815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2387fff640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23881f8090 0x7f238815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.746+0000 7f2387fff640 1 -- 172.21.15.105:0/2046264488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880cdec0 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.747+0000 7f2387fff640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23881f7410 0x7f23881f7890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2380009ed0 tx=0x7f2380010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.747+0000 7f236ffff640 1 -- 172.21.15.105:0/2046264488 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238001a030 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.747+0000 7f236ffff640 1 -- 172.21.15.105:0/2046264488 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2380010dd0 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.747+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238815cc10 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.747+0000 7f236ffff640 1 -- 172.21.15.105:0/2046264488 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380003bd0 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.747+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238815d1d0 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.748+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23880cf960 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.748+0000 7f236ffff640 1 -- 172.21.15.105:0/2046264488 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f238001d070 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.748+0000 7f236ffff640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f237406b500 0x7f237406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.749+0000 7f236ffff640 1 -- 172.21.15.105:0/2046264488 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f238008a650 con 0x7f23881f7410 2024-03-08T08:27:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.749+0000 7f23877fe640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f237406b500 0x7f237406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.749+0000 7f23877fe640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f237406b500 0x7f237406d9c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2378034000 tx=0x7f2378007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.754+0000 7f236ffff640 1 -- 172.21.15.105:0/2046264488 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23880cf960 con 0x7f23881f7410 2024-03-08T08:27:51.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.961+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2388068580 con 0x7f23881f7410 2024-03-08T08:27:51.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.961+0000 7f236ffff640 1 -- 172.21.15.105:0/2046264488 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2388068580 con 0x7f23881f7410 2024-03-08T08:27:51.963 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:51.964 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f237406b500 msgr2=0x7f237406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f237406b500 0x7f237406d9c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2378034000 tx=0x7f2378007000 comp rx=0 tx=0).stop 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23881f7410 msgr2=0x7f23881f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23881f7410 0x7f23881f7890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2380009ed0 tx=0x7f2380010590 comp rx=0 tx=0).stop 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 shutdown_connections 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f237406b500 0x7f237406d9c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23881f7410 0x7f23881f7890 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23881f8090 0x7f238815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 --2- 172.21.15.105:0/2046264488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23700039b0 0x7f2388146c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.963+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 >> 172.21.15.105:0/2046264488 conn(0x7f23880b37a0 msgr2=0x7f23880b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.964+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 shutdown_connections 2024-03-08T08:27:51.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:51.964+0000 7f238e084640 1 -- 172.21.15.105:0/2046264488 wait complete. 2024-03-08T08:27:51.976 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-08T08:27:51.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:52.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.074+0000 7f9dd99b5640 1 Processor -- start 2024-03-08T08:27:52.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.074+0000 7f9dd99b5640 1 -- start start 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd99b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 0x7f9dd406ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd99b5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd41457b0 con 0x7f9dd406c8b0 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd99b5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd41454d0 con 0x7f9dd4144a00 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd99b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd415c2c0 con 0x7f9dd4144f60 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd37fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9dd406c8b0 0x7f9dd40cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40390/0 (socket says 172.21.15.105:40390) 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd37fe640 1 -- 172.21.15.105:0/1330344356 learned_addr learned my addr 172.21.15.105:0/1330344356 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd27fc640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 0x7f9dd406ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495151112 0 0) 0x7f9dd41457b0 con 0x7f9dd406c8b0 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.075+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dbc003520 con 0x7f9dd406c8b0 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3378906669 0 0) 0x7f9dbc003520 con 0x7f9dd406c8b0 2024-03-08T08:27:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 msgr2=0x7f9dd406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 0x7f9dd406ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 >> v1:172.21.15.144:6789/0 conn(0x7f9dd4144a00 legacy=0x7f9dd415cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd415c5a0 con 0x7f9dd406c8b0 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f9dcc001670 con 0x7f9dd406c8b0 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 >> v1:172.21.15.105:6789/0 conn(0x7f9dd406c8b0 legacy=0x7f9dd40cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 0x7f9dbc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dbc003fb0 0x7f9dbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dbc004d40 0x7f9dbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd415c2c0 con 0x7f9dbc003fb0 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc00bbb0 con 0x7f9dbc004d40 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc00bdb0 con 0x7f9dd4144f60 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd27fc640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dbc003fb0 0x7f9dbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd2ffd640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 0x7f9dbc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd37fe640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dbc004d40 0x7f9dbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd27fc640 1 -- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 msgr2=0x7f9dbc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd27fc640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 0x7f9dbc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd27fc640 1 -- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dbc004d40 msgr2=0x7f9dbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd27fc640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dbc004d40 0x7f9dbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd27fc640 1 -- 172.21.15.105:0/1330344356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc003520 con 0x7f9dbc003fb0 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.076+0000 7f9dd2ffd640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dd4144f60 0x7f9dbc0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd27fc640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dbc003fb0 0x7f9dbc004800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ac0 tx=0x7f9dbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85c1f9eaed2ef62 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8039070 con 0x7f9dbc003fb0 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc802c6b0 con 0x7f9dbc003fb0 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd1ffb640 1 -- 172.21.15.105:0/1330344356 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8009690 con 0x7f9dbc003fb0 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd99b5640 1 -- 172.21.15.105:0/1330344356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dbc003fb0 msgr2=0x7f9dbc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd99b5640 1 --2- 172.21.15.105:0/1330344356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dbc003fb0 0x7f9dbc004800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009ac0 tx=0x7f9dbc005830 comp rx=0 tx=0).stop 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd37fe640 1 -- 172.21.15.105:0/1330344356 reap_dead start 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd99b5640 1 -- 172.21.15.105:0/1330344356 shutdown_connections 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd99b5640 1 -- 172.21.15.105:0/1330344356 >> 172.21.15.105:0/1330344356 conn(0x7f9dd406bd90 msgr2=0x7f9dd406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd99b5640 1 -- 172.21.15.105:0/1330344356 shutdown_connections 2024-03-08T08:27:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.077+0000 7f9dd99b5640 1 -- 172.21.15.105:0/1330344356 wait complete. 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd99b5640 1 Processor -- start 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd99b5640 1 -- start start 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd99b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dbc003fb0 0x7f9dd413e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd99b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd406e7c0 0x7f9dd406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd99b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd415e790 0x7f9dd415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd99b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40cf580 con 0x7f9dd406e7c0 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd99b5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd406ff10 con 0x7f9dd415e790 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd99b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40cf700 con 0x7f9dbc003fb0 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.079+0000 7f9dd27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd406e7c0 0x7f9dd406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd406e7c0 0x7f9dd406ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59768/0 (socket says 172.21.15.105:59768) 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd27fc640 1 -- 172.21.15.105:0/980297793 learned_addr learned my addr 172.21.15.105:0/980297793 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd27fc640 1 -- 172.21.15.105:0/980297793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dbc003fb0 msgr2=0x7f9dd413e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd27fc640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dbc003fb0 0x7f9dd413e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd27fc640 1 -- 172.21.15.105:0/980297793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd415e790 msgr2=0x7f9dd415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd27fc640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd415e790 0x7f9dd415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd27fc640 1 -- 172.21.15.105:0/980297793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd414bbe0 con 0x7f9dd406e7c0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd27fc640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd406e7c0 0x7f9dd406ec40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8000c00 tx=0x7f9dc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dbbfff640 1 -- 172.21.15.105:0/980297793 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8042020 con 0x7f9dd406e7c0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dbbfff640 1 -- 172.21.15.105:0/980297793 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc803a040 con 0x7f9dd406e7c0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd415f500 con 0x7f9dd406e7c0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dbbfff640 1 -- 172.21.15.105:0/980297793 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8039070 con 0x7f9dd406e7c0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.080+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd413ec00 con 0x7f9dd406e7c0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.081+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd4075120 con 0x7f9dd406e7c0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.081+0000 7f9dbbfff640 1 -- 172.21.15.105:0/980297793 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dc8045020 con 0x7f9dd406e7c0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.082+0000 7f9dbbfff640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9dc006b500 0x7f9dc006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.082+0000 7f9dd2ffd640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9dc006b500 0x7f9dc006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.082+0000 7f9dbbfff640 1 -- 172.21.15.105:0/980297793 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dc80ab570 con 0x7f9dd406e7c0 2024-03-08T08:27:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.082+0000 7f9dd2ffd640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9dc006b500 0x7f9dc006d9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4001d50 tx=0x7f9dc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.087+0000 7f9dbbfff640 1 -- 172.21.15.105:0/980297793 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dc804c070 con 0x7f9dd406e7c0 2024-03-08T08:27:52.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.258+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dd40727e0 con 0x7f9dd406e7c0 2024-03-08T08:27:52.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.259+0000 7f9dbbfff640 1 -- 172.21.15.105:0/980297793 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f9dc807a480 con 0x7f9dd406e7c0 2024-03-08T08:27:52.261 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:52.261 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:52.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9dc006b500 msgr2=0x7f9dc006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9dc006b500 0x7f9dc006d9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4001d50 tx=0x7f9dc4009040 comp rx=0 tx=0).stop 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd406e7c0 msgr2=0x7f9dd406ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd406e7c0 0x7f9dd406ec40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8000c00 tx=0x7f9dc8004730 comp rx=0 tx=0).stop 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 shutdown_connections 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9dc006b500 0x7f9dc006d9c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd415e790 0x7f9dd415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd406e7c0 0x7f9dd406ec40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 --2- 172.21.15.105:0/980297793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9dbc003fb0 0x7f9dd413e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 >> 172.21.15.105:0/980297793 conn(0x7f9dd406bd90 msgr2=0x7f9dd40cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 shutdown_connections 2024-03-08T08:27:52.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.262+0000 7f9dd99b5640 1 -- 172.21.15.105:0/980297793 wait complete. 2024-03-08T08:27:52.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:27:52.275 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:27:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff90df640 1 Processor -- start 2024-03-08T08:27:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff90df640 1 -- start start 2024-03-08T08:27:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff90df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff90df640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efff40b5c30 0x7efff415c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff90df640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff415f7b0 con 0x7efff415df30 2024-03-08T08:27:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff90df640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff415e710 con 0x7efff40b5c30 2024-03-08T08:27:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff90df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff415e9f0 con 0x7efff415bea0 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff359e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efff40b5c30 0x7efff415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7effebfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efff415df30 0x7efff40b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40396/0 (socket says 172.21.15.105:40396) 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7efff359e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efff40b5c30 0x7efff415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59772/0 (socket says 172.21.15.105:59772) 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.376+0000 7effebfff640 1 -- 172.21.15.105:0/2052532128 learned_addr learned my addr 172.21.15.105:0/2052532128 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.377+0000 7efff2d9d640 1 --2- 172.21.15.105:0/2052532128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.377+0000 7efff359e640 1 -- 172.21.15.105:0/2052532128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 msgr2=0x7efff40b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.377+0000 7efff359e640 1 --2- 172.21.15.105:0/2052532128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.377+0000 7efff359e640 1 -- 172.21.15.105:0/2052532128 >> v1:172.21.15.105:6789/0 conn(0x7efff415df30 legacy=0x7efff40b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.377+0000 7efff359e640 1 -- 172.21.15.105:0/2052532128 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff415ecd0 con 0x7efff40b5c30 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.377+0000 7efff259c640 1 -- 172.21.15.105:0/2052532128 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222611094 0 0) 0x7efff415f7b0 con 0x7efff415df30 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.377+0000 7efff2d9d640 1 --2- 172.21.15.105:0/2052532128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.377+0000 7efff359e640 1 --2- 172.21.15.105:0/2052532128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40b5c30 0x7efff415c280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effec009d90 tx=0x7effec02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b41d3fc3082f11b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff259c640 1 -- 172.21.15.105:0/2052532128 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec039070 con 0x7efff40b5c30 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff259c640 1 -- 172.21.15.105:0/2052532128 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec02cc50 con 0x7efff40b5c30 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff259c640 1 -- 172.21.15.105:0/2052532128 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec034440 con 0x7efff40b5c30 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff90df640 1 -- 172.21.15.105:0/2052532128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40b5c30 msgr2=0x7efff415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff90df640 1 --2- 172.21.15.105:0/2052532128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40b5c30 0x7efff415c280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effec009d90 tx=0x7effec02bea0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff90df640 1 -- 172.21.15.105:0/2052532128 shutdown_connections 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff90df640 1 --2- 172.21.15.105:0/2052532128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff40b5c30 0x7efff415c280 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff90df640 1 --2- 172.21.15.105:0/2052532128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff90df640 1 -- 172.21.15.105:0/2052532128 >> 172.21.15.105:0/2052532128 conn(0x7efff409aa00 msgr2=0x7efff409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff90df640 1 -- 172.21.15.105:0/2052532128 shutdown_connections 2024-03-08T08:27:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.378+0000 7efff90df640 1 -- 172.21.15.105:0/2052532128 wait complete. 2024-03-08T08:27:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.379+0000 7efff90df640 1 Processor -- start 2024-03-08T08:27:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.379+0000 7efff90df640 1 -- start start 2024-03-08T08:27:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff90df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efff40b5c30 0x7efff40cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff90df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff90df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff415df30 0x7efff415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff90df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff409cb00 con 0x7efff415df30 2024-03-08T08:27:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff90df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff415f290 con 0x7efff40b5c30 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff90df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff415e3b0 con 0x7efff415bea0 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff415df30 0x7efff415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff415df30 0x7efff415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59778/0 (socket says 172.21.15.105:59778) 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff359e640 1 -- 172.21.15.105:0/675647045 learned_addr learned my addr 172.21.15.105:0/675647045 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff2d9d640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efff40b5c30 0x7efff40cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7effebfff640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff359e640 1 -- 172.21.15.105:0/675647045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 msgr2=0x7efff40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff359e640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff359e640 1 -- 172.21.15.105:0/675647045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efff40b5c30 msgr2=0x7efff40cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.380+0000 7efff359e640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efff40b5c30 0x7efff40cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.381+0000 7efff359e640 1 -- 172.21.15.105:0/675647045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff40cd380 con 0x7efff415df30 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.381+0000 7efff359e640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff415df30 0x7efff415c280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7effec009eb0 tx=0x7effec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.381+0000 7effeb7fe640 1 -- 172.21.15.105:0/675647045 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec044020 con 0x7efff415df30 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.381+0000 7effeb7fe640 1 -- 172.21.15.105:0/675647045 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec02ca50 con 0x7efff415df30 2024-03-08T08:27:52.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.381+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff40cafd0 con 0x7efff415df30 2024-03-08T08:27:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.381+0000 7effeb7fe640 1 -- 172.21.15.105:0/675647045 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec039040 con 0x7efff415df30 2024-03-08T08:27:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.381+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff40cb590 con 0x7efff415df30 2024-03-08T08:27:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.382+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff4075120 con 0x7efff415df30 2024-03-08T08:27:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.382+0000 7effeb7fe640 1 -- 172.21.15.105:0/675647045 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effec047020 con 0x7efff415df30 2024-03-08T08:27:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.383+0000 7effeb7fe640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effd406b4d0 0x7effd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.383+0000 7effeb7fe640 1 -- 172.21.15.105:0/675647045 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effec0acc10 con 0x7efff415df30 2024-03-08T08:27:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.383+0000 7efff2d9d640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effd406b4d0 0x7effd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.383+0000 7efff2d9d640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effd406b4d0 0x7effd406d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7effe400a990 tx=0x7effe4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.388+0000 7effeb7fe640 1 -- 172.21.15.105:0/675647045 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7effec04e030 con 0x7efff415df30 2024-03-08T08:27:52.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.556+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efff40b31c0 con 0x7efff415df30 2024-03-08T08:27:52.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.556+0000 7effeb7fe640 1 -- 172.21.15.105:0/675647045 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7effec02cc20 con 0x7efff415df30 2024-03-08T08:27:52.558 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:27:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effd406b4d0 msgr2=0x7effd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effd406b4d0 0x7effd406d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7effe400a990 tx=0x7effe4008040 comp rx=0 tx=0).stop 2024-03-08T08:27:52.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff415df30 msgr2=0x7efff415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff415df30 0x7efff415c280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7effec009eb0 tx=0x7effec004a40 comp rx=0 tx=0).stop 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 shutdown_connections 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effd406b4d0 0x7effd406d990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efff415df30 0x7efff415c280 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efff415bea0 0x7efff40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 --2- 172.21.15.105:0/675647045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efff40b5c30 0x7efff40cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 >> 172.21.15.105:0/675647045 conn(0x7efff409aa00 msgr2=0x7efff406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 shutdown_connections 2024-03-08T08:27:52.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.558+0000 7efff90df640 1 -- 172.21.15.105:0/675647045 wait complete. 2024-03-08T08:27:52.571 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:27:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.672+0000 7fe1860ae640 1 Processor -- start 2024-03-08T08:27:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.672+0000 7fe1860ae640 1 -- start start 2024-03-08T08:27:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.672+0000 7fe1860ae640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe1801469e0 0x7fe18006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.672+0000 7fe1860ae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 0x7fe18006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.672+0000 7fe1860ae640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1800adcd0 con 0x7fe1800ad870 2024-03-08T08:27:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.672+0000 7fe1860ae640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18006deb0 con 0x7fe1801469e0 2024-03-08T08:27:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.672+0000 7fe1860ae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18006e190 con 0x7fe18006ac80 2024-03-08T08:27:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17f7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe1801469e0 0x7fe18006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 0x7fe18006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17effd640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe1800ad870 0x7fe18006d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:41000/0 (socket says 172.21.15.105:41000) 2024-03-08T08:27:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17f7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe1801469e0 0x7fe18006fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60026/0 (socket says 172.21.15.105:60026) 2024-03-08T08:27:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17effd640 1 -- 172.21.15.105:0/2645091341 learned_addr learned my addr 172.21.15.105:0/2645091341 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17f7fe640 1 -- 172.21.15.105:0/2645091341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 msgr2=0x7fe18006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17f7fe640 1 --2- 172.21.15.105:0/2645091341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 0x7fe18006b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17f7fe640 1 -- 172.21.15.105:0/2645091341 >> v1:172.21.15.144:6789/0 conn(0x7fe1800ad870 legacy=0x7fe18006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17f7fe640 1 -- 172.21.15.105:0/2645091341 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800b42f0 con 0x7fe1801469e0 2024-03-08T08:27:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.673+0000 7fe17e7fc640 1 -- 172.21.15.105:0/2645091341 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631926064 0 0) 0x7fe1800adcd0 con 0x7fe1800ad870 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.674+0000 7fe17f7fe640 1 --2- 172.21.15.105:0/2645091341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 0x7fe18006fd60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe170009d90 tx=0x7fe17002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e34ed3e50b4f6cc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.674+0000 7fe17e7fc640 1 -- 172.21.15.105:0/2645091341 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170039070 con 0x7fe1801469e0 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.674+0000 7fe17e7fc640 1 -- 172.21.15.105:0/2645091341 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17002cbc0 con 0x7fe1801469e0 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.674+0000 7fe17e7fc640 1 -- 172.21.15.105:0/2645091341 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170034440 con 0x7fe1801469e0 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.675+0000 7fe1860ae640 1 -- 172.21.15.105:0/2645091341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 msgr2=0x7fe18006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.675+0000 7fe1860ae640 1 --2- 172.21.15.105:0/2645091341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 0x7fe18006fd60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe170009d90 tx=0x7fe17002be10 comp rx=0 tx=0).stop 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.675+0000 7fe1860ae640 1 -- 172.21.15.105:0/2645091341 shutdown_connections 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.675+0000 7fe1860ae640 1 --2- 172.21.15.105:0/2645091341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 0x7fe18006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.675+0000 7fe1860ae640 1 --2- 172.21.15.105:0/2645091341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 0x7fe18006fd60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.675+0000 7fe1860ae640 1 -- 172.21.15.105:0/2645091341 >> 172.21.15.105:0/2645091341 conn(0x7fe1800b2670 msgr2=0x7fe1800b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.675+0000 7fe1860ae640 1 -- 172.21.15.105:0/2645091341 shutdown_connections 2024-03-08T08:27:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.675+0000 7fe1860ae640 1 -- 172.21.15.105:0/2645091341 wait complete. 2024-03-08T08:27:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.676+0000 7fe1860ae640 1 Processor -- start 2024-03-08T08:27:52.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.676+0000 7fe1860ae640 1 -- start start 2024-03-08T08:27:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe1860ae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 0x7fe18015d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe1860ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1800ad870 0x7fe1800cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe1860ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 0x7fe180141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe1860ae640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe180146dc0 con 0x7fe1800ad870 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe1860ae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18006e470 con 0x7fe1801469e0 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe1860ae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800b4af0 con 0x7fe18006ac80 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1800ad870 0x7fe1800cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1800ad870 0x7fe1800cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59782/0 (socket says 172.21.15.105:59782) 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 -- 172.21.15.105:0/1850942681 learned_addr learned my addr 172.21.15.105:0/1850942681 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:52.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17f7fe640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 0x7fe18015d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17ffff640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 0x7fe180141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 -- 172.21.15.105:0/1850942681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 msgr2=0x7fe18015d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 0x7fe18015d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 -- 172.21.15.105:0/1850942681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 msgr2=0x7fe180141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 0x7fe180141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 -- 172.21.15.105:0/1850942681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800cdf40 con 0x7fe1800ad870 2024-03-08T08:27:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe17effd640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1800ad870 0x7fe1800cfaf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c4d0 tx=0x7fe17400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.677+0000 7fe167fff640 1 -- 172.21.15.105:0/1850942681 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174018070 con 0x7fe1800ad870 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.678+0000 7fe167fff640 1 -- 172.21.15.105:0/1850942681 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe174007500 con 0x7fe1800ad870 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.678+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1800ce1e0 con 0x7fe1800ad870 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.678+0000 7fe167fff640 1 -- 172.21.15.105:0/1850942681 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174004a20 con 0x7fe1800ad870 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.678+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1800ce7a0 con 0x7fe1800ad870 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.678+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe180075120 con 0x7fe1800ad870 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.679+0000 7fe167fff640 1 -- 172.21.15.105:0/1850942681 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe17401b070 con 0x7fe1800ad870 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.679+0000 7fe167fff640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b4d0 0x7fe16c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.679+0000 7fe167fff640 1 -- 172.21.15.105:0/1850942681 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe174089450 con 0x7fe1800ad870 2024-03-08T08:27:52.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.679+0000 7fe17f7fe640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b4d0 0x7fe16c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.680+0000 7fe17f7fe640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b4d0 0x7fe16c06d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe170036000 tx=0x7fe170030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.684+0000 7fe167fff640 1 -- 172.21.15.105:0/1850942681 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe174057c80 con 0x7fe1800ad870 2024-03-08T08:27:52.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.855+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1800b2280 con 0x7fe1800ad870 2024-03-08T08:27:52.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.855+0000 7fe167fff640 1 -- 172.21.15.105:0/1850942681 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe17405bd20 con 0x7fe1800ad870 2024-03-08T08:27:52.856 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:27:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b4d0 msgr2=0x7fe16c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b4d0 0x7fe16c06d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe170036000 tx=0x7fe170030000 comp rx=0 tx=0).stop 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1800ad870 msgr2=0x7fe1800cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1800ad870 0x7fe1800cfaf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c4d0 tx=0x7fe17400c990 comp rx=0 tx=0).stop 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 shutdown_connections 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b4d0 0x7fe16c06d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1801469e0 0x7fe180141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1800ad870 0x7fe1800cfaf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 --2- 172.21.15.105:0/1850942681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18006ac80 0x7fe18015d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 >> 172.21.15.105:0/1850942681 conn(0x7fe1800b2670 msgr2=0x7fe1800b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 shutdown_connections 2024-03-08T08:27:52.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.857+0000 7fe1860ae640 1 -- 172.21.15.105:0/1850942681 wait complete. 2024-03-08T08:27:52.869 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:27:52.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.970+0000 7f436c483640 1 Processor -- start 2024-03-08T08:27:52.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.970+0000 7f436c483640 1 -- start start 2024-03-08T08:27:52.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.971+0000 7f436c483640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4364161850 0x7f436406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.971+0000 7f436c483640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640d48a0 con 0x7f436406bcb0 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.971+0000 7f436c483640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640d4b80 con 0x7f4364161470 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.971+0000 7f436c483640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436406c170 con 0x7f4364161850 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.971+0000 7f436aa20640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4364161850 0x7f436406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.971+0000 7f4369a1e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f436406bcb0 0x7f43640b2450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:40402/0 (socket says 172.21.15.105:40402) 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.971+0000 7f4369a1e640 1 -- 172.21.15.105:0/2100739866 learned_addr learned my addr 172.21.15.105:0/2100739866 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.972+0000 7f436aa20640 1 -- 172.21.15.105:0/2100739866 >> v1:172.21.15.144:6789/0 conn(0x7f4364161470 legacy=0x7f436406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.972+0000 7f436aa20640 1 -- 172.21.15.105:0/2100739866 >> v1:172.21.15.105:6789/0 conn(0x7f436406bcb0 legacy=0x7f43640b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.972+0000 7f436aa20640 1 -- 172.21.15.105:0/2100739866 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436406c450 con 0x7f4364161850 2024-03-08T08:27:52.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.972+0000 7f436921d640 1 -- 172.21.15.105:0/2100739866 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598602129 0 0) 0x7f43640d48a0 con 0x7f436406bcb0 2024-03-08T08:27:52.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.972+0000 7f436aa20640 1 --2- 172.21.15.105:0/2100739866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364161850 0x7f436406c830 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f435c009d90 tx=0x7f435c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ca86ac99788fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436921d640 1 -- 172.21.15.105:0/2100739866 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c039070 con 0x7f4364161850 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436921d640 1 -- 172.21.15.105:0/2100739866 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c02cbd0 con 0x7f4364161850 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436921d640 1 -- 172.21.15.105:0/2100739866 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c034440 con 0x7f4364161850 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436c483640 1 -- 172.21.15.105:0/2100739866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364161850 msgr2=0x7f436406c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436c483640 1 --2- 172.21.15.105:0/2100739866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364161850 0x7f436406c830 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f435c009d90 tx=0x7f435c02be20 comp rx=0 tx=0).stop 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436c483640 1 -- 172.21.15.105:0/2100739866 shutdown_connections 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436c483640 1 --2- 172.21.15.105:0/2100739866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4364161850 0x7f436406c830 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436c483640 1 -- 172.21.15.105:0/2100739866 >> 172.21.15.105:0/2100739866 conn(0x7f43640d3ce0 msgr2=0x7f43640d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436c483640 1 -- 172.21.15.105:0/2100739866 shutdown_connections 2024-03-08T08:27:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.973+0000 7f436c483640 1 -- 172.21.15.105:0/2100739866 wait complete. 2024-03-08T08:27:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.974+0000 7f436c483640 1 Processor -- start 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.974+0000 7f436c483640 1 -- start start 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436c483640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f436406bcb0 0x7f43640c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436c483640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364161470 0x7f43640b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436c483640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4364161850 0x7f436415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436c483640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436406d4f0 con 0x7f436406bcb0 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436c483640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436406d370 con 0x7f4364161470 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436c483640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436406d670 con 0x7f4364161850 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436a21f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f436406bcb0 0x7f43640c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436a21f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f436406bcb0 0x7f43640c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59808/0 (socket says 172.21.15.105:59808) 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436a21f640 1 -- 172.21.15.105:0/3786548392 learned_addr learned my addr 172.21.15.105:0/3786548392 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436aa20640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4364161850 0x7f436415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f4369a1e640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364161470 0x7f43640b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436a21f640 1 -- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4364161850 msgr2=0x7f436415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436a21f640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4364161850 0x7f436415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436a21f640 1 -- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364161470 msgr2=0x7f43640b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436a21f640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364161470 0x7f43640b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f436a21f640 1 -- 172.21.15.105:0/3786548392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436415da00 con 0x7f436406bcb0 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.975+0000 7f4369a1e640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364161470 0x7f43640b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.976+0000 7f436a21f640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f436406bcb0 0x7f43640c7910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f436000c460 tx=0x7f436000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.976+0000 7f435affd640 1 -- 172.21.15.105:0/3786548392 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360016020 con 0x7f436406bcb0 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.976+0000 7f435affd640 1 -- 172.21.15.105:0/3786548392 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4360012870 con 0x7f436406bcb0 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.976+0000 7f435affd640 1 -- 172.21.15.105:0/3786548392 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360012c60 con 0x7f436406bcb0 2024-03-08T08:27:52.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.976+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43640ca720 con 0x7f436406bcb0 2024-03-08T08:27:52.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.976+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43640b3240 con 0x7f436406bcb0 2024-03-08T08:27:52.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.977+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4364075120 con 0x7f436406bcb0 2024-03-08T08:27:52.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.977+0000 7f435affd640 1 -- 172.21.15.105:0/3786548392 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4360019020 con 0x7f436406bcb0 2024-03-08T08:27:52.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.978+0000 7f435affd640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f434806b4d0 0x7f434806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:52.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.978+0000 7f4369a1e640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f434806b4d0 0x7f434806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:52.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.978+0000 7f435affd640 1 -- 172.21.15.105:0/3786548392 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4360089b20 con 0x7f436406bcb0 2024-03-08T08:27:52.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.978+0000 7f4369a1e640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f434806b4d0 0x7f434806d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43640c9ce0 tx=0x7f4354007c30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:52.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:52.982+0000 7f435affd640 1 -- 172.21.15.105:0/3786548392 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4360020070 con 0x7f436406bcb0 2024-03-08T08:27:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.151+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43640b1700 con 0x7f436406bcb0 2024-03-08T08:27:53.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.151+0000 7f435affd640 1 -- 172.21.15.105:0/3786548392 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4360058a30 con 0x7f436406bcb0 2024-03-08T08:27:53.152 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:27:53.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f434806b4d0 msgr2=0x7f434806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f434806b4d0 0x7f434806d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43640c9ce0 tx=0x7f4354007c30 comp rx=0 tx=0).stop 2024-03-08T08:27:53.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f436406bcb0 msgr2=0x7f43640c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f436406bcb0 0x7f43640c7910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f436000c460 tx=0x7f436000c920 comp rx=0 tx=0).stop 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 shutdown_connections 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f434806b4d0 0x7f434806d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4364161850 0x7f436415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4364161470 0x7f43640b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 --2- 172.21.15.105:0/3786548392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f436406bcb0 0x7f43640c7910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 >> 172.21.15.105:0/3786548392 conn(0x7f43640d3ce0 msgr2=0x7f436415f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 shutdown_connections 2024-03-08T08:27:53.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.153+0000 7f436c483640 1 -- 172.21.15.105:0/3786548392 wait complete. 2024-03-08T08:27:53.164 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:27:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.262+0000 7ff07f5a0640 1 Processor -- start 2024-03-08T08:27:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.262+0000 7ff07f5a0640 1 -- start start 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.262+0000 7ff07f5a0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff07806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.262+0000 7ff07f5a0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0780ad910 0x7ff0780d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07f5a0640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07815e380 con 0x7ff07815dfa0 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07f5a0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0780b5f60 con 0x7ff0780ad910 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07f5a0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780b6240 con 0x7ff07806b040 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07db3d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0780ad910 0x7ff0780d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07db3d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff0780ad910 0x7ff0780d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59820/0 (socket says 172.21.15.105:59820) 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07db3d640 1 -- 172.21.15.105:0/1329394990 learned_addr learned my addr 172.21.15.105:0/1329394990 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07cb3b640 1 --2- 172.21.15.105:0/1329394990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff07806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07db3d640 1 -- 172.21.15.105:0/1329394990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 msgr2=0x7ff07806c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07db3d640 1 --2- 172.21.15.105:0/1329394990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff07806c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07db3d640 1 -- 172.21.15.105:0/1329394990 >> v1:172.21.15.144:6789/0 conn(0x7ff07815dfa0 legacy=0x7ff0780ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07db3d640 1 -- 172.21.15.105:0/1329394990 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07806b720 con 0x7ff0780ad910 2024-03-08T08:27:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.263+0000 7ff07cb3b640 1 --2- 172.21.15.105:0/1329394990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff07806c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07db3d640 1 --2- 172.21.15.105:0/1329394990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780d0170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff068009e50 tx=0x7ff06802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647d7b1842448c42 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff077fff640 1 -- 172.21.15.105:0/1329394990 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06802ca70 con 0x7ff0780ad910 2024-03-08T08:27:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff077fff640 1 -- 172.21.15.105:0/1329394990 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06802cc10 con 0x7ff0780ad910 2024-03-08T08:27:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff077fff640 1 -- 172.21.15.105:0/1329394990 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0680343f0 con 0x7ff0780ad910 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07f5a0640 1 -- 172.21.15.105:0/1329394990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 msgr2=0x7ff0780d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/1329394990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780d0170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff068009e50 tx=0x7ff06802bed0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07f5a0640 1 -- 172.21.15.105:0/1329394990 shutdown_connections 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/1329394990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780d0170 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/1329394990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff07806c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07f5a0640 1 -- 172.21.15.105:0/1329394990 >> 172.21.15.105:0/1329394990 conn(0x7ff0780b50f0 msgr2=0x7ff0780b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07f5a0640 1 -- 172.21.15.105:0/1329394990 shutdown_connections 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.264+0000 7ff07f5a0640 1 -- 172.21.15.105:0/1329394990 wait complete. 2024-03-08T08:27:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07f5a0640 1 Processor -- start 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07f5a0640 1 -- start start 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07f5a0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff078099930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07f5a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07f5a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff07815dfa0 0x7ff07815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07f5a0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07806d000 con 0x7ff0780ad910 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07f5a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07806bce0 con 0x7ff07815dfa0 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07f5a0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07806d180 con 0x7ff07806b040 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07cb3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07cb3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59824/0 (socket says 172.21.15.105:59824) 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07cb3b640 1 -- 172.21.15.105:0/450501367 learned_addr learned my addr 172.21.15.105:0/450501367 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07db3d640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff07815dfa0 0x7ff07815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.266+0000 7ff07d33c640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff078099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff07cb3b640 1 -- 172.21.15.105:0/450501367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 msgr2=0x7ff078099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff07cb3b640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff078099930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff07cb3b640 1 -- 172.21.15.105:0/450501367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff07815dfa0 msgr2=0x7ff07815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff07cb3b640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff07815dfa0 0x7ff07815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff07cb3b640 1 -- 172.21.15.105:0/450501367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078099e70 con 0x7ff0780ad910 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff07cb3b640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780b2410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c7f0 tx=0x7ff06c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff075ffb640 1 -- 172.21.15.105:0/450501367 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c015070 con 0x7ff0780ad910 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff075ffb640 1 -- 172.21.15.105:0/450501367 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c007a30 con 0x7ff0780ad910 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0781f8130 con 0x7ff0780ad910 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff075ffb640 1 -- 172.21.15.105:0/450501367 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c007e00 con 0x7ff0780ad910 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.267+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0780b1930 con 0x7ff0780ad910 2024-03-08T08:27:53.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.268+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff078075120 con 0x7ff0780ad910 2024-03-08T08:27:53.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.269+0000 7ff075ffb640 1 -- 172.21.15.105:0/450501367 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff06c018070 con 0x7ff0780ad910 2024-03-08T08:27:53.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.269+0000 7ff075ffb640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff06406b4d0 0x7ff06406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.269+0000 7ff07d33c640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff06406b4d0 0x7ff06406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.269+0000 7ff075ffb640 1 -- 172.21.15.105:0/450501367 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff06c08a030 con 0x7ff0780ad910 2024-03-08T08:27:53.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.269+0000 7ff07d33c640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff06406b4d0 0x7ff06406d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff070006fd0 tx=0x7ff070009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.273+0000 7ff075ffb640 1 -- 172.21.15.105:0/450501367 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff06c0588f0 con 0x7ff0780ad910 2024-03-08T08:27:53.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.482+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff078071ae0 con 0x7ff0780ad910 2024-03-08T08:27:53.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.482+0000 7ff075ffb640 1 -- 172.21.15.105:0/450501367 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4944 (secure 0 0 0) 0x7ff06c05c990 con 0x7ff0780ad910 2024-03-08T08:27:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:53.483 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:27:38.998370+0000","modified":"2024-03-08T08:27:41.001533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T08:27:53.483 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-08T08:27:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff06406b4d0 msgr2=0x7ff06406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff06406b4d0 0x7ff06406d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff070006fd0 tx=0x7ff070009060 comp rx=0 tx=0).stop 2024-03-08T08:27:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 msgr2=0x7ff0780b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780b2410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c7f0 tx=0x7ff06c00cbf0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 shutdown_connections 2024-03-08T08:27:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff06406b4d0 0x7ff06406d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff07815dfa0 0x7ff07815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0780ad910 0x7ff0780b2410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 --2- 172.21.15.105:0/450501367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff07806b040 0x7ff078099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 >> 172.21.15.105:0/450501367 conn(0x7ff0780b50f0 msgr2=0x7ff0780ad200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 shutdown_connections 2024-03-08T08:27:53.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.484+0000 7ff07f5a0640 1 -- 172.21.15.105:0/450501367 wait complete. 2024-03-08T08:27:53.496 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:53.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.594+0000 7f0c77a2e640 1 Processor -- start 2024-03-08T08:27:53.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.594+0000 7f0c77a2e640 1 -- start start 2024-03-08T08:27:53.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c77a2e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c700cfdb0 0x7f0c700b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c77a2e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c700d06e0 0x7f0c7015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c77a2e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c700b2c20 con 0x7f0c700b2740 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c77a2e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c700af4c0 con 0x7f0c700d06e0 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c77a2e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700af7a0 con 0x7f0c700cfdb0 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c75fcb640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0c700b2740 0x7f0c7015f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47546/0 (socket says 172.21.15.105:47546) 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c75fcb640 1 -- 172.21.15.105:0/2117894225 learned_addr learned my addr 172.21.15.105:0/2117894225 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956370807 0 0) 0x7f0c700b2c20 con 0x7f0c700b2740 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c757ca640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c700cfdb0 0x7f0c700b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c5c003520 con 0x7f0c700b2740 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c74fc9640 1 --2- 172.21.15.105:0/2117894225 >> v2:172.21.15.144:3300/0 conn(0x7f0c700d06e0 0x7f0c7015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145124917 0 0) 0x7f0c5c003520 con 0x7f0c700b2740 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c700cfdb0 msgr2=0x7f0c700b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:53.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c700cfdb0 0x7f0c700b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 >> v2:172.21.15.144:3300/0 conn(0x7f0c700d06e0 msgr2=0x7f0c7015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 --2- 172.21.15.105:0/2117894225 >> v2:172.21.15.144:3300/0 conn(0x7f0c700d06e0 0x7f0c7015ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700cc490 con 0x7f0c700b2740 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f0c680019f0 con 0x7f0c700b2740 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 >> v1:172.21.15.105:6789/0 conn(0x7f0c700b2740 legacy=0x7f0c7015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c0039b0 0x7f0c5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c700cfdb0 0x7f0c5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c5c004d40 0x7f0c5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700af4c0 con 0x7f0c5c004d40 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700af7a0 con 0x7f0c5c0039b0 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.595+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c00bb20 con 0x7f0c700cfdb0 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c75fcb640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c5c004d40 0x7f0c5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c757ca640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c0039b0 0x7f0c5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c74fc9640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c700cfdb0 0x7f0c5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c75fcb640 1 -- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c700cfdb0 msgr2=0x7f0c5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c75fcb640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c700cfdb0 0x7f0c5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c75fcb640 1 -- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c0039b0 msgr2=0x7f0c5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c75fcb640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c0039b0 0x7f0c5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c75fcb640 1 -- 172.21.15.105:0/2117894225 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c003520 con 0x7f0c5c004d40 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.596+0000 7f0c75fcb640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c5c004d40 0x7f0c5c0051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c680013d0 tx=0x7f0c5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829444929df51b55 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800a4e0 con 0x7f0c5c004d40 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c68034650 con 0x7f0c5c004d40 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c67fff640 1 -- 172.21.15.105:0/2117894225 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68034a40 con 0x7f0c5c004d40 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c77a2e640 1 -- 172.21.15.105:0/2117894225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c5c004d40 msgr2=0x7f0c5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c77a2e640 1 --2- 172.21.15.105:0/2117894225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c5c004d40 0x7f0c5c0051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0c680013d0 tx=0x7f0c5c005830 comp rx=0 tx=0).stop 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c75fcb640 1 -- 172.21.15.105:0/2117894225 reap_dead start 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c77a2e640 1 -- 172.21.15.105:0/2117894225 shutdown_connections 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c77a2e640 1 -- 172.21.15.105:0/2117894225 >> 172.21.15.105:0/2117894225 conn(0x7f0c700b36a0 msgr2=0x7f0c700b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:53.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c77a2e640 1 -- 172.21.15.105:0/2117894225 shutdown_connections 2024-03-08T08:27:53.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.597+0000 7f0c77a2e640 1 -- 172.21.15.105:0/2117894225 wait complete. 2024-03-08T08:27:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.598+0000 7f0c77a2e640 1 Processor -- start 2024-03-08T08:27:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.598+0000 7f0c77a2e640 1 -- start start 2024-03-08T08:27:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c77a2e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c004d40 0x7f0c70157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c77a2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700b4600 0x7f0c7006cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c77a2e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c7006d460 0x7f0c701f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c77a2e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700d0310 con 0x7f0c700b4600 2024-03-08T08:27:53.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c77a2e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700d0190 con 0x7f0c5c004d40 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c77a2e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700d0490 con 0x7f0c7006d460 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c74fc9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700b4600 0x7f0c7006cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c74fc9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700b4600 0x7f0c7006cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37916/0 (socket says 172.21.15.105:37916) 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c74fc9640 1 -- 172.21.15.105:0/1941136875 learned_addr learned my addr 172.21.15.105:0/1941136875 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c75fcb640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c7006d460 0x7f0c701f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c757ca640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c004d40 0x7f0c70157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c74fc9640 1 -- 172.21.15.105:0/1941136875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c7006d460 msgr2=0x7f0c701f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c74fc9640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c7006d460 0x7f0c701f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c74fc9640 1 -- 172.21.15.105:0/1941136875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c004d40 msgr2=0x7f0c70157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.599+0000 7f0c74fc9640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c004d40 0x7f0c70157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.600+0000 7f0c74fc9640 1 -- 172.21.15.105:0/1941136875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70158460 con 0x7f0c700b4600 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.600+0000 7f0c74fc9640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700b4600 0x7f0c7006cf20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000b7f0 tx=0x7f0c6000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.600+0000 7f0c65ffb640 1 -- 172.21.15.105:0/1941136875 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60018070 con 0x7f0c700b4600 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.600+0000 7f0c65ffb640 1 -- 172.21.15.105:0/1941136875 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c60010830 con 0x7f0c700b4600 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.600+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7006d9f0 con 0x7f0c700b4600 2024-03-08T08:27:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.600+0000 7f0c65ffb640 1 -- 172.21.15.105:0/1941136875 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60010c00 con 0x7f0c700b4600 2024-03-08T08:27:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.600+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c700c66d0 con 0x7f0c700b4600 2024-03-08T08:27:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.601+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c70075120 con 0x7f0c700b4600 2024-03-08T08:27:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.601+0000 7f0c65ffb640 1 -- 172.21.15.105:0/1941136875 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c6001b070 con 0x7f0c700b4600 2024-03-08T08:27:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.602+0000 7f0c65ffb640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0c5406b4d0 0x7f0c5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.602+0000 7f0c757ca640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0c5406b4d0 0x7f0c5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.602+0000 7f0c65ffb640 1 -- 172.21.15.105:0/1941136875 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c6008a7d0 con 0x7f0c700b4600 2024-03-08T08:27:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.602+0000 7f0c757ca640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0c5406b4d0 0x7f0c5406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c006fd0 tx=0x7f0c6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.606+0000 7f0c65ffb640 1 -- 172.21.15.105:0/1941136875 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c600596e0 con 0x7f0c700b4600 2024-03-08T08:27:53.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.780+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c700cefc0 con 0x7f0c700b4600 2024-03-08T08:27:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.782+0000 7f0c65ffb640 1 -- 172.21.15.105:0/1941136875 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f0c60058fe0 con 0x7f0c700b4600 2024-03-08T08:27:53.783 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:53.783 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.783+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0c5406b4d0 msgr2=0x7f0c5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.783+0000 7f0c77a2e640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0c5406b4d0 0x7f0c5406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c006fd0 tx=0x7f0c6c008040 comp rx=0 tx=0).stop 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.783+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700b4600 msgr2=0x7f0c7006cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.783+0000 7f0c77a2e640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700b4600 0x7f0c7006cf20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000b7f0 tx=0x7f0c6000bcb0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.784+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 shutdown_connections 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.784+0000 7f0c77a2e640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0c5406b4d0 0x7f0c5406d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.784+0000 7f0c77a2e640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c7006d460 0x7f0c701f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.784+0000 7f0c77a2e640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c700b4600 0x7f0c7006cf20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.784+0000 7f0c77a2e640 1 --2- 172.21.15.105:0/1941136875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c5c004d40 0x7f0c70157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.784+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 >> 172.21.15.105:0/1941136875 conn(0x7f0c700b36a0 msgr2=0x7f0c700cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.784+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 shutdown_connections 2024-03-08T08:27:53.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.784+0000 7f0c77a2e640 1 -- 172.21.15.105:0/1941136875 wait complete. 2024-03-08T08:27:53.796 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:27:53.796 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:27:53.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.897+0000 7f91cd4d4640 1 Processor -- start 2024-03-08T08:27:53.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.897+0000 7f91cd4d4640 1 -- start start 2024-03-08T08:27:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91cd4d4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c815cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91cd4d4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f91c8144f60 0x7f91c806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91cd4d4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f91c806c8b0 0x7f91c806b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91cd4d4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c81457b0 con 0x7f91c8144f60 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91cd4d4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c81454d0 con 0x7f91c806c8b0 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91cd4d4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c815c2c0 con 0x7f91c8144a00 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91c67fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f91c8144f60 0x7f91c806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91c67fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f91c8144f60 0x7f91c806ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37918/0 (socket says 172.21.15.105:37918) 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91c67fc640 1 -- 172.21.15.105:0/3931522272 learned_addr learned my addr 172.21.15.105:0/3931522272 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91c6ffd640 1 --2- 172.21.15.105:0/3931522272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c815cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91c77fe640 1 --2- 172.21.15.105:0/3931522272 >> v2:172.21.15.144:3300/0 conn(0x7f91c806c8b0 0x7f91c806b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.898+0000 7f91c67fc640 1 -- 172.21.15.105:0/3931522272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 msgr2=0x7f91c815cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.899+0000 7f91c67fc640 1 --2- 172.21.15.105:0/3931522272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c815cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.899+0000 7f91c67fc640 1 -- 172.21.15.105:0/3931522272 >> v2:172.21.15.144:3300/0 conn(0x7f91c806c8b0 msgr2=0x7f91c806b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.899+0000 7f91c67fc640 1 --2- 172.21.15.105:0/3931522272 >> v2:172.21.15.144:3300/0 conn(0x7f91c806c8b0 0x7f91c806b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.899+0000 7f91c67fc640 1 -- 172.21.15.105:0/3931522272 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c815c5a0 con 0x7f91c8144f60 2024-03-08T08:27:53.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.899+0000 7f91c67fc640 1 --2- 172.21.15.105:0/3931522272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c8144f60 0x7f91c806ac80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91c80ce340 tx=0x7f91bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90aabddf30b132b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.899+0000 7f91c5ffb640 1 -- 172.21.15.105:0/3931522272 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc035020 con 0x7f91c8144f60 2024-03-08T08:27:53.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.899+0000 7f91c5ffb640 1 -- 172.21.15.105:0/3931522272 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc031870 con 0x7f91c8144f60 2024-03-08T08:27:53.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.899+0000 7f91c5ffb640 1 -- 172.21.15.105:0/3931522272 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc031c40 con 0x7f91c8144f60 2024-03-08T08:27:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 -- 172.21.15.105:0/3931522272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c8144f60 msgr2=0x7f91c806ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/3931522272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c8144f60 0x7f91c806ac80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91c80ce340 tx=0x7f91bc02b910 comp rx=0 tx=0).stop 2024-03-08T08:27:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 -- 172.21.15.105:0/3931522272 shutdown_connections 2024-03-08T08:27:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/3931522272 >> v2:172.21.15.144:3300/0 conn(0x7f91c806c8b0 0x7f91c806b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/3931522272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c8144f60 0x7f91c806ac80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/3931522272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c815cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 -- 172.21.15.105:0/3931522272 >> 172.21.15.105:0/3931522272 conn(0x7f91c806bd90 msgr2=0x7f91c806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 -- 172.21.15.105:0/3931522272 shutdown_connections 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.900+0000 7f91cd4d4640 1 -- 172.21.15.105:0/3931522272 wait complete. 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.901+0000 7f91cd4d4640 1 Processor -- start 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.901+0000 7f91cd4d4640 1 -- start start 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91cd4d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c806c8b0 0x7f91c813c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91cd4d4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c8160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91cd4d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c8144f60 0x7f91c8159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91cd4d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c806ff10 con 0x7f91c806c8b0 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91cd4d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c815d2e0 con 0x7f91c8144f60 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91cd4d4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80cee70 con 0x7f91c8144a00 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c806c8b0 0x7f91c813c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c806c8b0 0x7f91c813c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37920/0 (socket says 172.21.15.105:37920) 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 -- 172.21.15.105:0/2161799824 learned_addr learned my addr 172.21.15.105:0/2161799824 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c67fc640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c8160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c77fe640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c8144f60 0x7f91c8159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 -- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 msgr2=0x7f91c8160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c8160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 -- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c8144f60 msgr2=0x7f91c8159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c8144f60 0x7f91c8159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 -- 172.21.15.105:0/2161799824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80cc7d0 con 0x7f91c806c8b0 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c67fc640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c8160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.902+0000 7f91c6ffd640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c806c8b0 0x7f91c813c4e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91b800efc0 tx=0x7f91b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.903+0000 7f91af7fe640 1 -- 172.21.15.105:0/2161799824 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8018020 con 0x7f91c806c8b0 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.903+0000 7f91af7fe640 1 -- 172.21.15.105:0/2161799824 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b8014480 con 0x7f91c806c8b0 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.903+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c81433b0 con 0x7f91c806c8b0 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.903+0000 7f91af7fe640 1 -- 172.21.15.105:0/2161799824 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8014850 con 0x7f91c806c8b0 2024-03-08T08:27:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.903+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c8143970 con 0x7f91c806c8b0 2024-03-08T08:27:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.904+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c8075120 con 0x7f91c806c8b0 2024-03-08T08:27:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.904+0000 7f91af7fe640 1 -- 172.21.15.105:0/2161799824 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91b8005d10 con 0x7f91c806c8b0 2024-03-08T08:27:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.905+0000 7f91af7fe640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f91b406b4d0 0x7f91b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.905+0000 7f91c67fc640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f91b406b4d0 0x7f91b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.905+0000 7f91af7fe640 1 -- 172.21.15.105:0/2161799824 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91b808ace0 con 0x7f91c806c8b0 2024-03-08T08:27:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.905+0000 7f91c67fc640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f91b406b4d0 0x7f91b406d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91bc036000 tx=0x7f91bc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:53.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:53.909+0000 7f91af7fe640 1 -- 172.21.15.105:0/2161799824 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91b8022070 con 0x7f91c806c8b0 2024-03-08T08:27:54.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.078+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91c815bca0 con 0x7f91c806c8b0 2024-03-08T08:27:54.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.079+0000 7f91af7fe640 1 -- 172.21.15.105:0/2161799824 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91b8010040 con 0x7f91c806c8b0 2024-03-08T08:27:54.080 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:27:54.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f91b406b4d0 msgr2=0x7f91b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f91b406b4d0 0x7f91b406d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91bc036000 tx=0x7f91bc03c040 comp rx=0 tx=0).stop 2024-03-08T08:27:54.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c806c8b0 msgr2=0x7f91c813c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c806c8b0 0x7f91c813c4e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91b800efc0 tx=0x7f91b800c490 comp rx=0 tx=0).stop 2024-03-08T08:27:54.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 shutdown_connections 2024-03-08T08:27:54.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f91b406b4d0 0x7f91b406d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91c8144f60 0x7f91c8159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91c8144a00 0x7f91c8160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 --2- 172.21.15.105:0/2161799824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91c806c8b0 0x7f91c813c4e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 >> 172.21.15.105:0/2161799824 conn(0x7f91c806bd90 msgr2=0x7f91c806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 shutdown_connections 2024-03-08T08:27:54.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.081+0000 7f91cd4d4640 1 -- 172.21.15.105:0/2161799824 wait complete. 2024-03-08T08:27:54.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:27:54.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.194+0000 7f3f62023640 1 Processor -- start 2024-03-08T08:27:54.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.194+0000 7f3f62023640 1 -- start start 2024-03-08T08:27:54.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f62023640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3f5c0d2530 0x7f3f5c0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f62023640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d2910 0x7f3f5c15de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f62023640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c0d48f0 con 0x7f3f5c0d43e0 2024-03-08T08:27:54.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f62023640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0d4bb0 con 0x7f3f5c0d2530 2024-03-08T08:27:54.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f62023640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0ceb30 con 0x7f3f5c0d2910 2024-03-08T08:27:54.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3f5c0d2530 0x7f3f5c0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3f5c0d43e0 0x7f3f5c0b3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47548/0 (socket says 172.21.15.105:47548) 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5affd640 1 -- 172.21.15.105:0/285355107 learned_addr learned my addr 172.21.15.105:0/285355107 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5b7fe640 1 -- 172.21.15.105:0/285355107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d2910 msgr2=0x7f3f5c15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5bfff640 1 --2- 172.21.15.105:0/285355107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d2910 0x7f3f5c15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5b7fe640 1 --2- 172.21.15.105:0/285355107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d2910 0x7f3f5c15de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5b7fe640 1 -- 172.21.15.105:0/285355107 >> v1:172.21.15.105:6789/0 conn(0x7f3f5c0d43e0 legacy=0x7f3f5c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5a7fc640 1 -- 172.21.15.105:0/285355107 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117066634 0 0) 0x7f3f5c0d48f0 con 0x7f3f5c0d43e0 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.195+0000 7f3f5b7fe640 1 -- 172.21.15.105:0/285355107 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0cee10 con 0x7f3f5c0d2530 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.196+0000 7f3f5b7fe640 1 --2- 172.21.15.105:0/285355107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2530 0x7f3f5c0b5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0097a0 tx=0x7f3f4c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353a9a8ae0068714 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.196+0000 7f3f5a7fc640 1 -- 172.21.15.105:0/285355107 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0315a0 con 0x7f3f5c0d2530 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.196+0000 7f3f5a7fc640 1 -- 172.21.15.105:0/285355107 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c037070 con 0x7f3f5c0d2530 2024-03-08T08:27:54.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.196+0000 7f3f5a7fc640 1 -- 172.21.15.105:0/285355107 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c031960 con 0x7f3f5c0d2530 2024-03-08T08:27:54.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.197+0000 7f3f62023640 1 -- 172.21.15.105:0/285355107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2530 msgr2=0x7f3f5c0b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.197+0000 7f3f62023640 1 --2- 172.21.15.105:0/285355107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2530 0x7f3f5c0b5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0097a0 tx=0x7f3f4c02b840 comp rx=0 tx=0).stop 2024-03-08T08:27:54.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.197+0000 7f3f62023640 1 -- 172.21.15.105:0/285355107 shutdown_connections 2024-03-08T08:27:54.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.197+0000 7f3f62023640 1 --2- 172.21.15.105:0/285355107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d2910 0x7f3f5c15de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.197+0000 7f3f62023640 1 --2- 172.21.15.105:0/285355107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2530 0x7f3f5c0b5170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.197+0000 7f3f62023640 1 -- 172.21.15.105:0/285355107 >> 172.21.15.105:0/285355107 conn(0x7f3f5c09cfb0 msgr2=0x7f3f5c0d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.197+0000 7f3f62023640 1 -- 172.21.15.105:0/285355107 shutdown_connections 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.197+0000 7f3f62023640 1 -- 172.21.15.105:0/285355107 wait complete. 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.198+0000 7f3f62023640 1 Processor -- start 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.198+0000 7f3f62023640 1 -- start start 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f62023640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f5c0d2530 0x7f3f5c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f62023640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2910 0x7f3f5c0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f62023640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d43e0 0x7f3f5c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f62023640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c15e340 con 0x7f3f5c0d2910 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f62023640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0b3080 con 0x7f3f5c0d2530 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f62023640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c15e4c0 con 0x7f3f5c0d43e0 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2910 0x7f3f5c0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2910 0x7f3f5c0b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37944/0 (socket says 172.21.15.105:37944) 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 -- 172.21.15.105:0/2950659214 learned_addr learned my addr 172.21.15.105:0/2950659214 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5bfff640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d43e0 0x7f3f5c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5b7fe640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f5c0d2530 0x7f3f5c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 -- 172.21.15.105:0/2950659214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d43e0 msgr2=0x7f3f5c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d43e0 0x7f3f5c0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 -- 172.21.15.105:0/2950659214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f5c0d2530 msgr2=0x7f3f5c0c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f5c0d2530 0x7f3f5c0c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 -- 172.21.15.105:0/2950659214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0c8ae0 con 0x7f3f5c0d2910 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f5affd640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2910 0x7f3f5c0b3980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000c460 tx=0x7f3f5000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f43fff640 1 -- 172.21.15.105:0/2950659214 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50018070 con 0x7f3f5c0d2910 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f43fff640 1 -- 172.21.15.105:0/2950659214 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f50007500 con 0x7f3f5c0d2910 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c0cd540 con 0x7f3f5c0d2910 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f43fff640 1 -- 172.21.15.105:0/2950659214 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50004a20 con 0x7f3f5c0d2910 2024-03-08T08:27:54.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.199+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c0c7dc0 con 0x7f3f5c0d2910 2024-03-08T08:27:54.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.200+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c075120 con 0x7f3f5c0d2910 2024-03-08T08:27:54.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.201+0000 7f3f43fff640 1 -- 172.21.15.105:0/2950659214 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f5001b070 con 0x7f3f5c0d2910 2024-03-08T08:27:54.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.201+0000 7f3f43fff640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f4806b4d0 0x7f3f4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.201+0000 7f3f5b7fe640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f4806b4d0 0x7f3f4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.201+0000 7f3f43fff640 1 -- 172.21.15.105:0/2950659214 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f50089440 con 0x7f3f5c0d2910 2024-03-08T08:27:54.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.202+0000 7f3f5b7fe640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f4806b4d0 0x7f3f4806d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c036000 tx=0x7f3f4c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.205+0000 7f3f43fff640 1 -- 172.21.15.105:0/2950659214 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f5c0cd540 con 0x7f3f5c0d2910 2024-03-08T08:27:54.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.373+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f5c09d500 con 0x7f3f5c0d2910 2024-03-08T08:27:54.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.374+0000 7f3f43fff640 1 -- 172.21.15.105:0/2950659214 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f50058350 con 0x7f3f5c0d2910 2024-03-08T08:27:54.375 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:27:54.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f4806b4d0 msgr2=0x7f3f4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f4806b4d0 0x7f3f4806d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c036000 tx=0x7f3f4c03d040 comp rx=0 tx=0).stop 2024-03-08T08:27:54.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2910 msgr2=0x7f3f5c0b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2910 0x7f3f5c0b3980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000c460 tx=0x7f3f5000c920 comp rx=0 tx=0).stop 2024-03-08T08:27:54.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 shutdown_connections 2024-03-08T08:27:54.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3f4806b4d0 0x7f3f4806d990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f5c0d43e0 0x7f3f5c0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f5c0d2910 0x7f3f5c0b3980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 --2- 172.21.15.105:0/2950659214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f5c0d2530 0x7f3f5c0c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 >> 172.21.15.105:0/2950659214 conn(0x7f3f5c09cfb0 msgr2=0x7f3f5c0b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:54.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 shutdown_connections 2024-03-08T08:27:54.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.376+0000 7f3f62023640 1 -- 172.21.15.105:0/2950659214 wait complete. 2024-03-08T08:27:54.387 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:27:54.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.489+0000 7f0f25c35640 1 Processor -- start 2024-03-08T08:27:54.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.489+0000 7f0f25c35640 1 -- start start 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.490+0000 7f0f25c35640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0f20144f60 0x7f0f2006ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.490+0000 7f0f25c35640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f201457b0 con 0x7f0f2006c8b0 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.490+0000 7f0f25c35640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f201454d0 con 0x7f0f20144a00 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.490+0000 7f0f25c35640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2015c2c0 con 0x7f0f20144f60 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.490+0000 7f0f1ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0f2006c8b0 0x7f0f200cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47556/0 (socket says 172.21.15.105:47556) 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.490+0000 7f0f1ffff640 1 -- 172.21.15.105:0/22384960 learned_addr learned my addr 172.21.15.105:0/22384960 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.490+0000 7f0f1effd640 1 --2- 172.21.15.105:0/22384960 >> v2:172.21.15.144:3300/0 conn(0x7f0f20144f60 0x7f0f2006ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.491+0000 7f0f1e7fc640 1 -- 172.21.15.105:0/22384960 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320554561 0 0) 0x7f0f201457b0 con 0x7f0f2006c8b0 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.491+0000 7f0f1e7fc640 1 -- 172.21.15.105:0/22384960 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f08003520 con 0x7f0f2006c8b0 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.491+0000 7f0f1effd640 1 -- 172.21.15.105:0/22384960 >> v1:172.21.15.144:6789/0 conn(0x7f0f20144a00 legacy=0x7f0f2015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.491+0000 7f0f1e7fc640 1 -- 172.21.15.105:0/22384960 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996189095 0 0) 0x7f0f08003520 con 0x7f0f2006c8b0 2024-03-08T08:27:54.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.491+0000 7f0f1effd640 1 -- 172.21.15.105:0/22384960 >> v1:172.21.15.105:6789/0 conn(0x7f0f2006c8b0 legacy=0x7f0f200cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.491+0000 7f0f1effd640 1 -- 172.21.15.105:0/22384960 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2015c5a0 con 0x7f0f20144f60 2024-03-08T08:27:54.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f1effd640 1 --2- 172.21.15.105:0/22384960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 0x7f0f2006ac80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0f140097a0 tx=0x7f0f1402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebdda44b9927e466 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f1e7fc640 1 -- 172.21.15.105:0/22384960 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14037070 con 0x7f0f20144f60 2024-03-08T08:27:54.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f1e7fc640 1 -- 172.21.15.105:0/22384960 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f14030510 con 0x7f0f20144f60 2024-03-08T08:27:54.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f1e7fc640 1 -- 172.21.15.105:0/22384960 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f180019f0 con 0x7f0f20144f60 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f25c35640 1 -- 172.21.15.105:0/22384960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 msgr2=0x7f0f2006ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f25c35640 1 --2- 172.21.15.105:0/22384960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 0x7f0f2006ac80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0f140097a0 tx=0x7f0f1402b790 comp rx=0 tx=0).stop 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f25c35640 1 -- 172.21.15.105:0/22384960 shutdown_connections 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f25c35640 1 --2- 172.21.15.105:0/22384960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 0x7f0f2006ac80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f25c35640 1 -- 172.21.15.105:0/22384960 >> 172.21.15.105:0/22384960 conn(0x7f0f2006bd90 msgr2=0x7f0f2006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f25c35640 1 -- 172.21.15.105:0/22384960 shutdown_connections 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.492+0000 7f0f25c35640 1 -- 172.21.15.105:0/22384960 wait complete. 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.493+0000 7f0f25c35640 1 Processor -- start 2024-03-08T08:27:54.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f25c35640 1 -- start start 2024-03-08T08:27:54.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f25c35640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f2006c8b0 0x7f0f20155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f25c35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f20144a00 0x7f0f2015f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f25c35640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 0x7f0f2015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f25c35640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2006b1c0 con 0x7f0f20144a00 2024-03-08T08:27:54.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f25c35640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2006ff10 con 0x7f0f20144f60 2024-03-08T08:27:54.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f25c35640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2006b340 con 0x7f0f2006c8b0 2024-03-08T08:27:54.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f1effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f20144a00 0x7f0f2015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f1effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f20144a00 0x7f0f2015f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37950/0 (socket says 172.21.15.105:37950) 2024-03-08T08:27:54.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.494+0000 7f0f1effd640 1 -- 172.21.15.105:0/718652019 learned_addr learned my addr 172.21.15.105:0/718652019 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:54.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1ffff640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 0x7f0f2015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1f7fe640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f2006c8b0 0x7f0f20155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1effd640 1 -- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f2006c8b0 msgr2=0x7f0f20155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1effd640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f2006c8b0 0x7f0f20155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1effd640 1 -- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 msgr2=0x7f0f2015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1effd640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 0x7f0f2015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1effd640 1 -- 172.21.15.105:0/718652019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2015fe20 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1f7fe640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f2006c8b0 0x7f0f20155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f1effd640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f20144a00 0x7f0f2015f440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0f1402b7d0 tx=0x7f0f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f07fff640 1 -- 172.21.15.105:0/718652019 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14042070 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f07fff640 1 -- 172.21.15.105:0/718652019 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1403e870 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f20160100 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f07fff640 1 -- 172.21.15.105:0/718652019 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14037040 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.495+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2015f980 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.496+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f20160620 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.496+0000 7f0f07fff640 1 -- 172.21.15.105:0/718652019 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f14045070 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.497+0000 7f0f07fff640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.497+0000 7f0f07fff640 1 -- 172.21.15.105:0/718652019 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f140ac060 con 0x7f0f20144a00 2024-03-08T08:27:54.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.497+0000 7f0f1f7fe640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.497+0000 7f0f1f7fe640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0f2006fbd0 tx=0x7f0f10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.501+0000 7f0f07fff640 1 -- 172.21.15.105:0/718652019 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f1407aa40 con 0x7f0f20144a00 2024-03-08T08:27:54.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.673+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f20075120 con 0x7f0f20144a00 2024-03-08T08:27:54.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.673+0000 7f0f07fff640 1 -- 172.21.15.105:0/718652019 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f1407eae0 con 0x7f0f20144a00 2024-03-08T08:27:54.674 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0f0c06b4d0 msgr2=0x7f0f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0f2006fbd0 tx=0x7f0f10009040 comp rx=0 tx=0).stop 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f20144a00 msgr2=0x7f0f2015f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f20144a00 0x7f0f2015f440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0f1402b7d0 tx=0x7f0f140046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 shutdown_connections 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f20144f60 0x7f0f2015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f20144a00 0x7f0f2015f440 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 --2- 172.21.15.105:0/718652019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f2006c8b0 0x7f0f20155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 >> 172.21.15.105:0/718652019 conn(0x7f0f2006bd90 msgr2=0x7f0f2015cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 shutdown_connections 2024-03-08T08:27:54.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.675+0000 7f0f25c35640 1 -- 172.21.15.105:0/718652019 wait complete. 2024-03-08T08:27:54.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:27:54.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.786+0000 7f767e639640 1 Processor -- start 2024-03-08T08:27:54.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.786+0000 7f767e639640 1 -- start start 2024-03-08T08:27:54.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f767e639640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815df20 0x7f76780ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f767e639640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76781627c0 con 0x7f76781621a0 2024-03-08T08:27:54.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f767e639640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767806f070 con 0x7f76780aea80 2024-03-08T08:27:54.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f767e639640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767806ee20 con 0x7f767815df20 2024-03-08T08:27:54.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f767cbd6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f76781621a0 0x7f767806fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47560/0 (socket says 172.21.15.105:47560) 2024-03-08T08:27:54.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f767cbd6640 1 -- 172.21.15.105:0/2453731364 learned_addr learned my addr 172.21.15.105:0/2453731364 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:54.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724195299 0 0) 0x7f76781627c0 con 0x7f76781621a0 2024-03-08T08:27:54.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f7677fff640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815df20 0x7f76780ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7660003520 con 0x7f76781621a0 2024-03-08T08:27:54.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 325236282 0 0) 0x7f7660003520 con 0x7f76781621a0 2024-03-08T08:27:54.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815df20 msgr2=0x7f76780ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f7676ffd640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815df20 0x7f76780ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 >> v1:172.21.15.144:6789/0 conn(0x7f76780aea80 legacy=0x7f7678161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.787+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767806f350 con 0x7f76781621a0 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f76700019f0 con 0x7f76781621a0 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 >> v1:172.21.15.105:6789/0 conn(0x7f76781621a0 legacy=0x7f767806fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7676ffd640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76600039e0 0x7f7660003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7676ffd640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815df20 0x7f7660004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7676ffd640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7660004d40 0x7f76600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767806ee20 con 0x7f7660004d40 2024-03-08T08:27:54.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766000ba70 con 0x7f76600039e0 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766000bc70 con 0x7f767815df20 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f767cbd6640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7660004d40 0x7f76600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f7677fff640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76600039e0 0x7f7660003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f76777fe640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815df20 0x7f7660004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f767cbd6640 1 -- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815df20 msgr2=0x7f7660004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f767cbd6640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815df20 0x7f7660004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f767cbd6640 1 -- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76600039e0 msgr2=0x7f7660003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f767cbd6640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76600039e0 0x7f7660003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.788+0000 7f767cbd6640 1 -- 172.21.15.105:0/2453731364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660003520 con 0x7f7660004d40 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f767cbd6640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7660004d40 0x7f76600051c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7670002230 tx=0x7f7660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f781fa208e94bee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767000a4a0 con 0x7f7660004d40 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670039070 con 0x7f7660004d40 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f7676ffd640 1 -- 172.21.15.105:0/2453731364 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670035c30 con 0x7f7660004d40 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f767e639640 1 -- 172.21.15.105:0/2453731364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7660004d40 msgr2=0x7f76600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f767e639640 1 --2- 172.21.15.105:0/2453731364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7660004d40 0x7f76600051c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7670002230 tx=0x7f7660005830 comp rx=0 tx=0).stop 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f767cbd6640 1 -- 172.21.15.105:0/2453731364 reap_dead start 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f767e639640 1 -- 172.21.15.105:0/2453731364 shutdown_connections 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f767e639640 1 -- 172.21.15.105:0/2453731364 >> 172.21.15.105:0/2453731364 conn(0x7f767809aa00 msgr2=0x7f767809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f767e639640 1 -- 172.21.15.105:0/2453731364 shutdown_connections 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.789+0000 7f767e639640 1 -- 172.21.15.105:0/2453731364 wait complete. 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f767e639640 1 Processor -- start 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f767e639640 1 -- start start 2024-03-08T08:27:54.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f767e639640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7678156b50 0x7f7678160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f767e639640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678157780 0x7f767815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f767e639640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815f2e0 0x7f76781f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f767e639640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76780703e0 con 0x7f7678157780 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f767e639640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7678070260 con 0x7f7678156b50 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f767e639640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7678070560 con 0x7f767815f2e0 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.791+0000 7f76777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678157780 0x7f767815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f76777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678157780 0x7f767815eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37964/0 (socket says 172.21.15.105:37964) 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f76777fe640 1 -- 172.21.15.105:0/1162214982 learned_addr learned my addr 172.21.15.105:0/1162214982 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f7677fff640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7678156b50 0x7f7678160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f767cbd6640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815f2e0 0x7f76781f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f76777fe640 1 -- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815f2e0 msgr2=0x7f76781f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f76777fe640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815f2e0 0x7f76781f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f76777fe640 1 -- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7678156b50 msgr2=0x7f7678160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f76777fe640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7678156b50 0x7f7678160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:54.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f76777fe640 1 -- 172.21.15.105:0/1162214982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76781613b0 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f767cbd6640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815f2e0 0x7f76781f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f76777fe640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678157780 0x7f767815eda0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f766c00f820 tx=0x7f766c00fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f7674ff9640 1 -- 172.21.15.105:0/1162214982 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c0050d0 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f7674ff9640 1 -- 172.21.15.105:0/1162214982 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c018070 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7678158000 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f7674ff9640 1 -- 172.21.15.105:0/1162214982 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c00d6d0 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.792+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76781585c0 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.793+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7678075120 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.794+0000 7f7674ff9640 1 -- 172.21.15.105:0/1162214982 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f766c00d870 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.794+0000 7f7674ff9640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f766406b500 0x7f766406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.794+0000 7f7674ff9640 1 -- 172.21.15.105:0/1162214982 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f766c089f90 con 0x7f7678157780 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.794+0000 7f7677fff640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f766406b500 0x7f766406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:54.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.794+0000 7f7677fff640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f766406b500 0x7f766406d9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7668009bc0 tx=0x7f7668009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:54.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:54.798+0000 7f7674ff9640 1 -- 172.21.15.105:0/1162214982 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f766c058d80 con 0x7f7678157780 2024-03-08T08:27:55.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.004+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f767815f6c0 con 0x7f7678157780 2024-03-08T08:27:55.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.005+0000 7f7674ff9640 1 -- 172.21.15.105:0/1162214982 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4944 (secure 0 0 0) 0x7f766c058680 con 0x7f7678157780 2024-03-08T08:27:55.006 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:55.006 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:27:38.998370+0000","modified":"2024-03-08T08:27:41.001533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T08:27:55.006 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-08T08:27:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f766406b500 msgr2=0x7f766406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f766406b500 0x7f766406d9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7668009bc0 tx=0x7f7668009190 comp rx=0 tx=0).stop 2024-03-08T08:27:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678157780 msgr2=0x7f767815eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678157780 0x7f767815eda0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f766c00f820 tx=0x7f766c00fc20 comp rx=0 tx=0).stop 2024-03-08T08:27:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 shutdown_connections 2024-03-08T08:27:55.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f766406b500 0x7f766406d9c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f767815f2e0 0x7f76781f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678157780 0x7f767815eda0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 --2- 172.21.15.105:0/1162214982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7678156b50 0x7f7678160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 >> 172.21.15.105:0/1162214982 conn(0x7f767809aa00 msgr2=0x7f767815d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:55.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 shutdown_connections 2024-03-08T08:27:55.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.007+0000 7f767e639640 1 -- 172.21.15.105:0/1162214982 wait complete. 2024-03-08T08:27:55.019 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:27:55.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.118+0000 7f696f0c9640 1 Processor -- start 2024-03-08T08:27:55.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.118+0000 7f696f0c9640 1 -- start start 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f696f0c9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f696815df20 0x7f696806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f696f0c9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69680b5990 0x7f696806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f696f0c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696806f830 con 0x7f696806b1c0 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f696f0c9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696806dbb0 con 0x7f696815df20 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f696f0c9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696806b5f0 con 0x7f69680b5990 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f696ce65640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f696815df20 0x7f696806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f696d666640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f696806b1c0 0x7f69680b3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47576/0 (socket says 172.21.15.105:47576) 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f696d666640 1 -- 172.21.15.105:0/3327684090 learned_addr learned my addr 172.21.15.105:0/3327684090 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.119+0000 7f6967fff640 1 --2- 172.21.15.105:0/3327684090 >> v2:172.21.15.144:3300/0 conn(0x7f69680b5990 0x7f696806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.120+0000 7f696ce65640 1 -- 172.21.15.105:0/3327684090 >> v2:172.21.15.144:3300/0 conn(0x7f69680b5990 msgr2=0x7f696806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.120+0000 7f696ce65640 1 --2- 172.21.15.105:0/3327684090 >> v2:172.21.15.144:3300/0 conn(0x7f69680b5990 0x7f696806ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.120+0000 7f696ce65640 1 -- 172.21.15.105:0/3327684090 >> v1:172.21.15.105:6789/0 conn(0x7f696806b1c0 legacy=0x7f69680b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.120+0000 7f69677fe640 1 -- 172.21.15.105:0/3327684090 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237354853 0 0) 0x7f696806f830 con 0x7f696806b1c0 2024-03-08T08:27:55.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.120+0000 7f696ce65640 1 -- 172.21.15.105:0/3327684090 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696806fad0 con 0x7f696815df20 2024-03-08T08:27:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.120+0000 7f696ce65640 1 --2- 172.21.15.105:0/3327684090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696815df20 0x7f696806c8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6968072b00 tx=0x7f695802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c940ec746788e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.120+0000 7f69677fe640 1 -- 172.21.15.105:0/3327684090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580315a0 con 0x7f696815df20 2024-03-08T08:27:55.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f69677fe640 1 -- 172.21.15.105:0/3327684090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6958037070 con 0x7f696815df20 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f69677fe640 1 -- 172.21.15.105:0/3327684090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958031960 con 0x7f696815df20 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f696f0c9640 1 -- 172.21.15.105:0/3327684090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696815df20 msgr2=0x7f696806c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3327684090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696815df20 0x7f696806c8b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6968072b00 tx=0x7f695802b840 comp rx=0 tx=0).stop 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f696f0c9640 1 -- 172.21.15.105:0/3327684090 shutdown_connections 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3327684090 >> v2:172.21.15.144:3300/0 conn(0x7f69680b5990 0x7f696806ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3327684090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696815df20 0x7f696806c8b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f696f0c9640 1 -- 172.21.15.105:0/3327684090 >> 172.21.15.105:0/3327684090 conn(0x7f696806bd90 msgr2=0x7f696806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f696f0c9640 1 -- 172.21.15.105:0/3327684090 shutdown_connections 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.121+0000 7f696f0c9640 1 -- 172.21.15.105:0/3327684090 wait complete. 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.122+0000 7f696f0c9640 1 Processor -- start 2024-03-08T08:27:55.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.122+0000 7f696f0c9640 1 -- start start 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696f0c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696806b1c0 0x7f696813fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696f0c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69680b5990 0x7f6968153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696f0c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f696815df20 0x7f69681539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696f0c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696806e0e0 con 0x7f696806b1c0 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696f0c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696806de00 con 0x7f69680b5990 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696f0c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69680ae980 con 0x7f696815df20 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696ce65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696806b1c0 0x7f696813fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696ce65640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696806b1c0 0x7f696813fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37982/0 (socket says 172.21.15.105:37982) 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696ce65640 1 -- 172.21.15.105:0/3551199478 learned_addr learned my addr 172.21.15.105:0/3551199478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f6967fff640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69680b5990 0x7f6968153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696d666640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f696815df20 0x7f69681539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.123+0000 7f696ce65640 1 -- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f696815df20 msgr2=0x7f69681539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f696ce65640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f696815df20 0x7f69681539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f696ce65640 1 -- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69680b5990 msgr2=0x7f6968153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f696ce65640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69680b5990 0x7f6968153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f696ce65640 1 -- 172.21.15.105:0/3551199478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6968151ea0 con 0x7f696806b1c0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f696d666640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f696815df20 0x7f69681539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f6967fff640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69680b5990 0x7f6968153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f696ce65640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696806b1c0 0x7f696813fd60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6958034040 tx=0x7f69580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f69657fa640 1 -- 172.21.15.105:0/3551199478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958004230 con 0x7f696806b1c0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f69657fa640 1 -- 172.21.15.105:0/3551199478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580043d0 con 0x7f696806b1c0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696815ba10 con 0x7f696806b1c0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f69657fa640 1 -- 172.21.15.105:0/3551199478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958003850 con 0x7f696806b1c0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.124+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696815bfd0 con 0x7f696806b1c0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.125+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6968075120 con 0x7f696806b1c0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.125+0000 7f69657fa640 1 -- 172.21.15.105:0/3551199478 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69580039f0 con 0x7f696806b1c0 2024-03-08T08:27:55.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.126+0000 7f69657fa640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f695406b4d0 0x7f695406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.126+0000 7f6967fff640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f695406b4d0 0x7f695406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.126+0000 7f69657fa640 1 -- 172.21.15.105:0/3551199478 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f695802f030 con 0x7f696806b1c0 2024-03-08T08:27:55.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.126+0000 7f6967fff640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f695406b4d0 0x7f695406d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f696815e420 tx=0x7f695c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:55.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.130+0000 7f69657fa640 1 -- 172.21.15.105:0/3551199478 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6958041070 con 0x7f696806b1c0 2024-03-08T08:27:55.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.302+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69680d07a0 con 0x7f696806b1c0 2024-03-08T08:27:55.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.304+0000 7f69657fa640 1 -- 172.21.15.105:0/3551199478 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f695807b280 con 0x7f696806b1c0 2024-03-08T08:27:55.305 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:55.305 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:27:55.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f695406b4d0 msgr2=0x7f695406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f695406b4d0 0x7f695406d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f696815e420 tx=0x7f695c008040 comp rx=0 tx=0).stop 2024-03-08T08:27:55.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696806b1c0 msgr2=0x7f696813fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696806b1c0 0x7f696813fd60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6958034040 tx=0x7f69580046c0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 shutdown_connections 2024-03-08T08:27:55.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f695406b4d0 0x7f695406d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f696815df20 0x7f69681539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69680b5990 0x7f6968153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 --2- 172.21.15.105:0/3551199478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f696806b1c0 0x7f696813fd60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 >> 172.21.15.105:0/3551199478 conn(0x7f696806bd90 msgr2=0x7f69680b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:55.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 shutdown_connections 2024-03-08T08:27:55.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.306+0000 7f696f0c9640 1 -- 172.21.15.105:0/3551199478 wait complete. 2024-03-08T08:27:55.318 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.417+0000 7f1711bfd640 1 Processor -- start 2024-03-08T08:27:55.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.417+0000 7f1711bfd640 1 -- start start 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f1711bfd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f170c0d06e0 0x7f170c15f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f1711bfd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f170c0b2740 0x7f170c0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f1711bfd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c0b2c20 con 0x7f170c0cfdb0 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f1711bfd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c0af4c0 con 0x7f170c0d06e0 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f1711bfd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c0af7a0 con 0x7f170c0b2740 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f170c0d06e0 0x7f170c15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f170c0cfdb0 0x7f170c15ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47578/0 (socket says 172.21.15.105:47578) 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170b7fe640 1 -- 172.21.15.105:0/645650265 learned_addr learned my addr 172.21.15.105:0/645650265 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170bfff640 1 --2- 172.21.15.105:0/645650265 >> v2:172.21.15.144:3300/0 conn(0x7f170c0b2740 0x7f170c0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170affd640 1 -- 172.21.15.105:0/645650265 >> v2:172.21.15.144:3300/0 conn(0x7f170c0b2740 msgr2=0x7f170c0b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170affd640 1 --2- 172.21.15.105:0/645650265 >> v2:172.21.15.144:3300/0 conn(0x7f170c0b2740 0x7f170c0b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170affd640 1 -- 172.21.15.105:0/645650265 >> v1:172.21.15.105:6789/0 conn(0x7f170c0cfdb0 legacy=0x7f170c15ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170a7fc640 1 -- 172.21.15.105:0/645650265 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956212320 0 0) 0x7f170c0b2c20 con 0x7f170c0cfdb0 2024-03-08T08:27:55.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170affd640 1 -- 172.21.15.105:0/645650265 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0cc490 con 0x7f170c0d06e0 2024-03-08T08:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.418+0000 7f170bfff640 1 --2- 172.21.15.105:0/645650265 >> v2:172.21.15.144:3300/0 conn(0x7f170c0b2740 0x7f170c0b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f170affd640 1 --2- 172.21.15.105:0/645650265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15f580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f170c0d18b0 tx=0x7f170002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e0f8cfa85c7ebc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f170a7fc640 1 -- 172.21.15.105:0/645650265 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17000315a0 con 0x7f170c0d06e0 2024-03-08T08:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f170a7fc640 1 -- 172.21.15.105:0/645650265 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1700037070 con 0x7f170c0d06e0 2024-03-08T08:27:55.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f170a7fc640 1 -- 172.21.15.105:0/645650265 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700031960 con 0x7f170c0d06e0 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f1711bfd640 1 -- 172.21.15.105:0/645650265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 msgr2=0x7f170c15f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f1711bfd640 1 --2- 172.21.15.105:0/645650265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15f580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f170c0d18b0 tx=0x7f170002b840 comp rx=0 tx=0).stop 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f1711bfd640 1 -- 172.21.15.105:0/645650265 shutdown_connections 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f1711bfd640 1 --2- 172.21.15.105:0/645650265 >> v2:172.21.15.144:3300/0 conn(0x7f170c0b2740 0x7f170c0b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f1711bfd640 1 --2- 172.21.15.105:0/645650265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15f580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f1711bfd640 1 -- 172.21.15.105:0/645650265 >> 172.21.15.105:0/645650265 conn(0x7f170c0b36a0 msgr2=0x7f170c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f1711bfd640 1 -- 172.21.15.105:0/645650265 shutdown_connections 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.419+0000 7f1711bfd640 1 -- 172.21.15.105:0/645650265 wait complete. 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.421+0000 7f1711bfd640 1 Processor -- start 2024-03-08T08:27:55.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.421+0000 7f1711bfd640 1 -- start start 2024-03-08T08:27:55.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.421+0000 7f1711bfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f170c0b2740 0x7f170c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.421+0000 7f1711bfd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f170c0cfdb0 0x7f170c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.421+0000 7f1711bfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.421+0000 7f1711bfd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c165b70 con 0x7f170c0d06e0 2024-03-08T08:27:55.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.421+0000 7f1711bfd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0d03a0 con 0x7f170c0b2740 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.421+0000 7f1711bfd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c165cf0 con 0x7f170c0cfdb0 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38012/0 (socket says 172.21.15.105:38012) 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 -- 172.21.15.105:0/4149280022 learned_addr learned my addr 172.21.15.105:0/4149280022 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 -- 172.21.15.105:0/4149280022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f170c0cfdb0 msgr2=0x7f170c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170b7fe640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f170c0b2740 0x7f170c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f170c0cfdb0 0x7f170c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 -- 172.21.15.105:0/4149280022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f170c0b2740 msgr2=0x7f170c153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f170c0b2740 0x7f170c153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 -- 172.21.15.105:0/4149280022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0c8190 con 0x7f170c0d06e0 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f170bfff640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15d810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f170400cd20 tx=0x7f170400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f16f3fff640 1 -- 172.21.15.105:0/4149280022 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400f680 con 0x7f170c0d06e0 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f16f3fff640 1 -- 172.21.15.105:0/4149280022 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170400f840 con 0x7f170c0d06e0 2024-03-08T08:27:55.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f16f3fff640 1 -- 172.21.15.105:0/4149280022 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400fc30 con 0x7f170c0d06e0 2024-03-08T08:27:55.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c161400 con 0x7f170c0d06e0 2024-03-08T08:27:55.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.422+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c1619c0 con 0x7f170c0d06e0 2024-03-08T08:27:55.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.423+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c161bc0 con 0x7f170c0d06e0 2024-03-08T08:27:55.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.423+0000 7f16f3fff640 1 -- 172.21.15.105:0/4149280022 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17040213f0 con 0x7f170c0d06e0 2024-03-08T08:27:55.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.424+0000 7f16f3fff640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16f806b4d0 0x7f16f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:55.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.424+0000 7f16f3fff640 1 -- 172.21.15.105:0/4149280022 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f170408aeb0 con 0x7f170c0d06e0 2024-03-08T08:27:55.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.424+0000 7f170b7fe640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16f806b4d0 0x7f16f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:55.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.424+0000 7f170b7fe640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16f806b4d0 0x7f16f806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16fc000f60 tx=0x7f16fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:55.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.428+0000 7f16f3fff640 1 -- 172.21.15.105:0/4149280022 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f170c161bc0 con 0x7f170c0d06e0 2024-03-08T08:27:55.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.633+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f170c075120 con 0x7f170c0d06e0 2024-03-08T08:27:55.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.633+0000 7f16f3fff640 1 -- 172.21.15.105:0/4149280022 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4944 (secure 0 0 0) 0x7f1704014030 con 0x7f170c0d06e0 2024-03-08T08:27:55.634 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:27:55.635 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:27:38.998370+0000","modified":"2024-03-08T08:27:41.001533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T08:27:55.635 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16f806b4d0 msgr2=0x7f16f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16f806b4d0 0x7f16f806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16fc000f60 tx=0x7f16fc009040 comp rx=0 tx=0).stop 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 msgr2=0x7f170c15d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15d810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f170400cd20 tx=0x7f170400a650 comp rx=0 tx=0).stop 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 shutdown_connections 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16f806b4d0 0x7f16f806d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f170c0d06e0 0x7f170c15d810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f170c0cfdb0 0x7f170c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.635+0000 7f1711bfd640 1 --2- 172.21.15.105:0/4149280022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f170c0b2740 0x7f170c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:55.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.636+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 >> 172.21.15.105:0/4149280022 conn(0x7f170c0b36a0 msgr2=0x7f170c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:55.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.636+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 shutdown_connections 2024-03-08T08:27:55.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:55.636+0000 7f1711bfd640 1 -- 172.21.15.105:0/4149280022 wait complete. 2024-03-08T08:27:56.049 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-08T08:27:38.998370+0000', 'modified': '2024-03-08T08:27:41.001533+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4278}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4278': {'gid': 4278, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6835/3724471877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6833', 'nonce': 3724471877}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3724471877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T08:27:56.050 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-08T08:27:56.050 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-08T08:27:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.151+0000 7fe6134f4640 1 Processor -- start 2024-03-08T08:27:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.151+0000 7fe6134f4640 1 -- start start 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe6134f4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe60c0b1550 0x7fe60c0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe6134f4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60c0b1af0 con 0x7fe60c15dd80 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe6134f4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60c0b1dd0 con 0x7fe60c0b2b90 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe6134f4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60c0b2700 con 0x7fe60c0b1550 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe610a8f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe60c0b1550 0x7fe60c0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe611290640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe60c15dd80 0x7fe60c1623c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47592/0 (socket says 172.21.15.105:47592) 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe611290640 1 -- 172.21.15.105:0/703749722 learned_addr learned my addr 172.21.15.105:0/703749722 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe610a8f640 1 -- 172.21.15.105:0/703749722 >> v1:172.21.15.144:6789/0 conn(0x7fe60c0b2b90 legacy=0x7fe60c0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe610a8f640 1 -- 172.21.15.105:0/703749722 >> v1:172.21.15.105:6789/0 conn(0x7fe60c15dd80 legacy=0x7fe60c1623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.152+0000 7fe610a8f640 1 -- 172.21.15.105:0/703749722 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c0b20b0 con 0x7fe60c0b1550 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe610a8f640 1 --2- 172.21.15.105:0/703749722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c0b1550 0x7fe60c0aea80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0097d0 tx=0x7fe5fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdc6057730463dd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe603fff640 1 -- 172.21.15.105:0/703749722 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc035020 con 0x7fe60c0b1550 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe603fff640 1 -- 172.21.15.105:0/703749722 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc031830 con 0x7fe60c0b1550 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe603fff640 1 -- 172.21.15.105:0/703749722 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc031c00 con 0x7fe60c0b1550 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe6134f4640 1 -- 172.21.15.105:0/703749722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c0b1550 msgr2=0x7fe60c0aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe6134f4640 1 --2- 172.21.15.105:0/703749722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c0b1550 0x7fe60c0aea80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0097d0 tx=0x7fe5fc02b8d0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe6134f4640 1 -- 172.21.15.105:0/703749722 shutdown_connections 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe6134f4640 1 --2- 172.21.15.105:0/703749722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c0b1550 0x7fe60c0aea80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.153+0000 7fe6134f4640 1 -- 172.21.15.105:0/703749722 >> 172.21.15.105:0/703749722 conn(0x7fe60c0b37a0 msgr2=0x7fe60c0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.154+0000 7fe6134f4640 1 -- 172.21.15.105:0/703749722 shutdown_connections 2024-03-08T08:27:56.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.154+0000 7fe6134f4640 1 -- 172.21.15.105:0/703749722 wait complete. 2024-03-08T08:27:56.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe6134f4640 1 Processor -- start 2024-03-08T08:27:56.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe6134f4640 1 -- start start 2024-03-08T08:27:56.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe6134f4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe60c0b1550 0x7fe60c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe6134f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe60c0b2b90 0x7fe60c1f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe6134f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c15dd80 0x7fe60c0cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe6134f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c15e160 con 0x7fe60c15dd80 2024-03-08T08:27:56.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe6134f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0aefc0 con 0x7fe60c0b2b90 2024-03-08T08:27:56.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe6134f4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0cc0d0 con 0x7fe60c0b1550 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.155+0000 7fe611a91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c15dd80 0x7fe60c0cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611a91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c15dd80 0x7fe60c0cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38036/0 (socket says 172.21.15.105:38036) 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611a91640 1 -- 172.21.15.105:0/4062716335 learned_addr learned my addr 172.21.15.105:0/4062716335 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe610a8f640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe60c0b2b90 0x7fe60c1f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611290640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe60c0b1550 0x7fe60c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611a91640 1 -- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe60c0b1550 msgr2=0x7fe60c151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611a91640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe60c0b1550 0x7fe60c151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611a91640 1 -- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe60c0b2b90 msgr2=0x7fe60c1f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611a91640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe60c0b2b90 0x7fe60c1f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611a91640 1 -- 172.21.15.105:0/4062716335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c0cc3b0 con 0x7fe60c15dd80 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611290640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe60c0b1550 0x7fe60c151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe610a8f640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe60c0b2b90 0x7fe60c1f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:56.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe611a91640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c15dd80 0x7fe60c0cd170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe60400c780 tx=0x7fe60400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe601ffb640 1 -- 172.21.15.105:0/4062716335 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604017070 con 0x7fe60c15dd80 2024-03-08T08:27:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe601ffb640 1 -- 172.21.15.105:0/4062716335 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604003da0 con 0x7fe60c15dd80 2024-03-08T08:27:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe6134f4640 1 -- 172.21.15.105:0/4062716335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60c0cc690 con 0x7fe60c15dd80 2024-03-08T08:27:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe601ffb640 1 -- 172.21.15.105:0/4062716335 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040052b0 con 0x7fe60c15dd80 2024-03-08T08:27:56.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.156+0000 7fe6134f4640 1 -- 172.21.15.105:0/4062716335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60c0ccc50 con 0x7fe60c15dd80 2024-03-08T08:27:56.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.158+0000 7fe601ffb640 1 -- 172.21.15.105:0/4062716335 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe60401a070 con 0x7fe60c15dd80 2024-03-08T08:27:56.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.158+0000 7fe601ffb640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5f006b4d0 0x7fe5f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.158+0000 7fe601ffb640 1 -- 172.21.15.105:0/4062716335 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe604089840 con 0x7fe60c15dd80 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.158+0000 7fe611290640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5f006b4d0 0x7fe5f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.159+0000 7fe611290640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5f006b4d0 0x7fe5f006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe60c1f7830 tx=0x7fe608009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.159+0000 7fe6161a9740 1 Processor -- start 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.159+0000 7fe6161a9740 1 -- start start 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c51870 0x55e8c3c51c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c558c0 0x55e8c3c57360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c81a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe6161a9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3bbb6b0 con 0x55e8c3c81600 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe6161a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3b64a60 con 0x55e8c3c51870 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe6161a9740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3bb7750 con 0x55e8c3c558c0 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c81a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c81a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38048/0 (socket says 172.21.15.105:38048) 2024-03-08T08:27:56.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 -- 172.21.15.105:0/3400481438 learned_addr learned my addr 172.21.15.105:0/3400481438 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611290640 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c51870 0x55e8c3c51c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe610a8f640 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c558c0 0x55e8c3c57360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 -- 172.21.15.105:0/3400481438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c558c0 msgr2=0x55e8c3c57360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c558c0 0x55e8c3c57360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 -- 172.21.15.105:0/3400481438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c51870 msgr2=0x55e8c3c51c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c51870 0x55e8c3c51c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 -- 172.21.15.105:0/3400481438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8c3b82f30 con 0x55e8c3c81600 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe611a91640 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c81a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe6040601a0 tx=0x7fe604058080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c722bd7a57199dd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe5f6ffd640 1 -- 172.21.15.105:0/3400481438 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604086070 con 0x55e8c3c81600 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe5f6ffd640 1 -- 172.21.15.105:0/3400481438 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6040562c0 con 0x55e8c3c81600 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.160+0000 7fe5f6ffd640 1 -- 172.21.15.105:0/3400481438 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60406f070 con 0x55e8c3c81600 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 -- 172.21.15.105:0/3400481438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 msgr2=0x55e8c3c81a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c81a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe6040601a0 tx=0x7fe604058080 comp rx=0 tx=0).stop 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 -- 172.21.15.105:0/3400481438 shutdown_connections 2024-03-08T08:27:56.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c81a80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c558c0 0x55e8c3c57360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 --2- 172.21.15.105:0/3400481438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c51870 0x55e8c3c51c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 -- 172.21.15.105:0/3400481438 >> 172.21.15.105:0/3400481438 conn(0x55e8c3c538f0 msgr2=0x55e8c3c551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 -- 172.21.15.105:0/3400481438 shutdown_connections 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.161+0000 7fe6161a9740 1 -- 172.21.15.105:0/3400481438 wait complete. 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.162+0000 7fe6161a9740 1 -- start start 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.162+0000 7fe6161a9740 1 Processor -- start 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c78f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9a390 0x55e8c3c9ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3cafd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3c55d50 con 0x55e8c3c81600 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3b77ea0 con 0x55e8c3c9cf90 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3c56030 con 0x55e8c3c9a390 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c78f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 10 client.0 _ll_register_callbacks cb 0x55e8c3b77870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c78f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38062/0 (socket says 172.21.15.105:38062) 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 -- 172.21.15.105:0/2768069168 learned_addr learned my addr 172.21.15.105:0/2768069168 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611a91640 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3cafd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe610a8f640 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9a390 0x55e8c3c9ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 -- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9a390 msgr2=0x55e8c3c9ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9a390 0x55e8c3c9ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 -- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 msgr2=0x55e8c3cafd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3cafd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 -- 172.21.15.105:0/2768069168 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8c3b78270 con 0x55e8c3c81600 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611a91640 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3cafd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe611290640 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c78f50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe608007350 tx=0x7fe608000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47b952b69122dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe5ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2768069168 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608026060 con 0x55e8c3c81600 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2768069168 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe608021a90 con 0x55e8c3c81600 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 1 -- 172.21.15.105:0/2768069168 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6040562c0 con 0x55e8c3c81600 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2768069168 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe608021cd0 con 0x55e8c3c81600 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 10 client.4668 fetch_fsmap learned FSMap version 6 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe6161a9740 1 -- 172.21.15.105:0/2768069168 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55e8c38d02d0 con 0x55e8c3c81600 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.163+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2768069168 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6040562c0 con 0x55e8c3c81600 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2768069168 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe608012410 con 0x55e8c3c81600 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe6161a9740 10 client.4668 fetch_fsmap finished waiting for FSMap version 6 2024-03-08T08:27:56.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe6161a9740 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe6161a9740 20 client.4668 populate_metadata read hostname 'smithi105' 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] conn(0x55e8c3c7de70 0x55e8c3cbb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe6161a9740 4 client.4668 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe6161a9740 1 -- 172.21.15.105:0/2768069168 --> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8c3bc8bf0 con 0x55e8c3c7de70 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe610a8f640 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] conn(0x55e8c3c7de70 0x55e8c3cbb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe610a8f640 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] conn(0x55e8c3c7de70 0x55e8c3cbb1a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24119e657163ac2 server_cookie=c161d8cd783ce91b in_seq=0 out_seq=0 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.164+0000 7fe5ebfff640 10 client.4668 ms_handle_connect on v2:172.21.15.144:6833/3724471877 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2768069168 <== mds.0 v2:172.21.15.144:6833/3724471877 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8c3bc8bf0 con 0x55e8c3c7de70 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe5ebfff640 10 client.4668 handle_command_reply: tid=0 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 client.4668 shutdown 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 -- 172.21.15.105:0/2768069168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 msgr2=0x55e8c3c78f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c78f50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe608007350 tx=0x7fe608000bc0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 -- 172.21.15.105:0/2768069168 shutdown_connections 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] conn(0x55e8c3c7de70 0x55e8c3cbb1a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3cafd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9a390 0x55e8c3c9ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2768069168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3c78f50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 -- 172.21.15.105:0/2768069168 >> 172.21.15.105:0/2768069168 conn(0x55e8c3c51870 msgr2=0x55e8c3c7f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe5ebfff640 0 client.4668 ms_handle_reset on v2:172.21.15.144:6833/3724471877 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 -- 172.21.15.105:0/2768069168 shutdown_connections 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 1 -- 172.21.15.105:0/2768069168 wait complete. 2024-03-08T08:27:56.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.165+0000 7fe6161a9740 20 client.4668 trim_cache size 0 max 16384 2024-03-08T08:27:56.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.181+0000 7fe6161a9740 1 Processor -- start 2024-03-08T08:27:56.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.181+0000 7fe6161a9740 1 -- start start 2024-03-08T08:27:56.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.181+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c81600 0x55e8c3c9fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.181+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c9a390 0x55e8c3c80d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.181+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3c7f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.181+0000 7fe6161a9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0b5490 con 0x55e8c3c9a390 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.181+0000 7fe6161a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0b4890 con 0x55e8c3c9cf90 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.181+0000 7fe6161a9740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0b4e90 con 0x55e8c3c81600 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c9a390 0x55e8c3c80d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c9a390 0x55e8c3c80d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38076/0 (socket says 172.21.15.105:38076) 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 -- 172.21.15.105:0/548768636 learned_addr learned my addr 172.21.15.105:0/548768636 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe611290640 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c81600 0x55e8c3c9fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 -- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c81600 msgr2=0x55e8c3c9fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe611a91640 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3c7f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c81600 0x55e8c3c9fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 -- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 msgr2=0x55e8c3c7f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3c7f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 -- 172.21.15.105:0/548768636 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8c399a6b0 con 0x55e8c3c9a390 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe611290640 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c81600 0x55e8c3c9fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe611a91640 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3c7f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe610a8f640 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c9a390 0x55e8c3c80d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc034040 tx=0x7fe5fc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43123cf3387742a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe5f67fc640 1 -- 172.21.15.105:0/548768636 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc031d20 con 0x55e8c3c9a390 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe5f67fc640 1 -- 172.21.15.105:0/548768636 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc047420 con 0x55e8c3c9a390 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.182+0000 7fe5f67fc640 1 -- 172.21.15.105:0/548768636 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc047810 con 0x55e8c3c9a390 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 -- 172.21.15.105:0/548768636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c9a390 msgr2=0x55e8c3c80d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c9a390 0x55e8c3c80d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc034040 tx=0x7fe5fc030000 comp rx=0 tx=0).stop 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 -- 172.21.15.105:0/548768636 shutdown_connections 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9cf90 0x55e8c3c7f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c9a390 0x55e8c3c80d60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 --2- 172.21.15.105:0/548768636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c81600 0x55e8c3c9fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 -- 172.21.15.105:0/548768636 >> 172.21.15.105:0/548768636 conn(0x55e8c3c51870 msgr2=0x55e8c3c55080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:56.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 -- 172.21.15.105:0/548768636 shutdown_connections 2024-03-08T08:27:56.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 -- 172.21.15.105:0/548768636 wait complete. 2024-03-08T08:27:56.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 -- start start 2024-03-08T08:27:56.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.183+0000 7fe6161a9740 1 Processor -- start 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3d068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9a390 0x55e8c3c7f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe6161a9740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9cf90 0x55e8c3c7fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe6161a9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3c7a0f0 con 0x55e8c3c81600 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe6161a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3c7a740 con 0x55e8c3c9a390 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe6161a9740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8c3c7e760 con 0x55e8c3c9cf90 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe6161a9740 10 client.0 _ll_register_callbacks cb 0x7fe608001c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611290640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3d068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611290640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3d068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38082/0 (socket says 172.21.15.105:38082) 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611290640 1 -- 172.21.15.105:0/2560744804 learned_addr learned my addr 172.21.15.105:0/2560744804 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:56.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe610a8f640 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9a390 0x55e8c3c7f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611a91640 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9cf90 0x55e8c3c7fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611290640 1 -- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9cf90 msgr2=0x55e8c3c7fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611290640 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9cf90 0x55e8c3c7fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611290640 1 -- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9a390 msgr2=0x55e8c3c7f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611290640 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9a390 0x55e8c3c7f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611290640 1 -- 172.21.15.105:0/2560744804 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8c3c9cb60 con 0x55e8c3c81600 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.184+0000 7fe611a91640 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9cf90 0x55e8c3c7fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe611290640 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3d068c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6080011f0 tx=0x7fe608007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e46689427df4dab server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe5ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2560744804 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60803b060 con 0x55e8c3c81600 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2560744804 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe60801d040 con 0x55e8c3c81600 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 1 -- 172.21.15.105:0/2560744804 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5fc047420 con 0x55e8c3c81600 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2560744804 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe608001340 con 0x55e8c3c81600 2024-03-08T08:27:56.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 10 client.4674 fetch_fsmap learned FSMap version 6 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 1 -- 172.21.15.105:0/2560744804 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55e8c3d072a0 con 0x55e8c3c81600 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2560744804 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc047420 con 0x55e8c3c81600 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2560744804 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe608012b20 con 0x55e8c3c81600 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 10 client.4674 fetch_fsmap finished waiting for FSMap version 6 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 10 client.4674 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 20 client.4674 populate_metadata read hostname 'smithi105' 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] conn(0x55e8c3cbfb20 0x55e8c3c50510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 4 client.4674 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe6161a9740 1 -- 172.21.15.105:0/2560744804 --> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e8c3c52630 con 0x55e8c3cbfb20 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.185+0000 7fe610a8f640 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] conn(0x55e8c3cbfb20 0x55e8c3c50510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe5ebfff640 10 client.4674 ms_handle_connect on v2:172.21.15.144:6833/3724471877 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe610a8f640 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] conn(0x55e8c3cbfb20 0x55e8c3c50510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ee5ce13271c75c server_cookie=f9dac3a02b5b8052 in_seq=0 out_seq=0 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe5ebfff640 1 -- 172.21.15.105:0/2560744804 <== mds.0 v2:172.21.15.144:6833/3724471877 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55e8c3c52630 con 0x55e8c3cbfb20 2024-03-08T08:27:56.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe5ebfff640 10 client.4674 handle_command_reply: tid=0 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 client.4674 shutdown 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 -- 172.21.15.105:0/2560744804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 msgr2=0x55e8c3d068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3d068c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe6080011f0 tx=0x7fe608007000 comp rx=0 tx=0).stop 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 -- 172.21.15.105:0/2560744804 shutdown_connections 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:6833/3724471877,v1:172.21.15.144:6835/3724471877] conn(0x55e8c3cbfb20 0x55e8c3c50510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8c3c9cf90 0x55e8c3c7fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8c3c9a390 0x55e8c3c7f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe5ebfff640 0 client.4674 ms_handle_reset on v2:172.21.15.144:6833/3724471877 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 --2- 172.21.15.105:0/2560744804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8c3c81600 0x55e8c3d068c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 -- 172.21.15.105:0/2560744804 >> 172.21.15.105:0/2560744804 conn(0x55e8c3c51870 msgr2=0x55e8c3caf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 -- 172.21.15.105:0/2560744804 shutdown_connections 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 1 -- 172.21.15.105:0/2560744804 wait complete. 2024-03-08T08:27:56.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.186+0000 7fe6161a9740 20 client.4674 trim_cache size 0 max 16384 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "1afff3d7-46c3-4cd2-9212-c89ee069e495", 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4278, 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 16.526237337000001, 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 6, 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 20, 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 20, 2024-03-08T08:27:56.195 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 23.552079801000001 2024-03-08T08:27:56.196 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:27:56.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 -- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5f006b4d0 msgr2=0x7fe5f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5f006b4d0 0x7fe5f006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe60c1f7830 tx=0x7fe608009040 comp rx=0 tx=0).stop 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 -- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c15dd80 msgr2=0x7fe60c0cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c15dd80 0x7fe60c0cd170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe60400c780 tx=0x7fe60400cc40 comp rx=0 tx=0).stop 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 -- 172.21.15.105:0/4062716335 shutdown_connections 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe5f006b4d0 0x7fe5f006d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe60c15dd80 0x7fe60c0cd170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe60c0b2b90 0x7fe60c1f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 --2- 172.21.15.105:0/4062716335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe60c0b1550 0x7fe60c151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 -- 172.21.15.105:0/4062716335 >> 172.21.15.105:0/4062716335 conn(0x7fe60c0b37a0 msgr2=0x7fe60c06f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 -- 172.21.15.105:0/4062716335 shutdown_connections 2024-03-08T08:27:56.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.187+0000 7fe6134f4640 1 -- 172.21.15.105:0/4062716335 wait complete. 2024-03-08T08:27:56.198 INFO:teuthology.run_tasks:Running task kclient... 2024-03-08T08:27:56.221 INFO:tasks.kclient:Mounting kernel clients... 2024-03-08T08:27:56.221 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-08T08:27:56.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.222 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-08T08:27:56.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-08T08:27:56.305 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-03-08T08:27:56.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:27:56.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.306 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-08T08:27:56.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.361 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-08T08:27:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-03-08T08:27:56.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:27:56.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.431 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-03-08T08:27:56.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.448 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-03-08T08:27:56.521 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2024-03-08T08:27:56.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:27:56.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.523 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-03-08T08:27:56.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.580 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-03-08T08:27:56.651 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2024-03-08T08:27:56.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:27:56.653 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-08T08:27:56.653 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T08:27:56.653 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T08:27:56.653 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T08:27:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.753+0000 7f58ff393640 1 Processor -- start 2024-03-08T08:27:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.754+0000 7f58ff393640 1 -- start start 2024-03-08T08:27:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.754+0000 7f58ff393640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80b4000 0x7f58f815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.754+0000 7f58ff393640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58f806cf90 0x7f58f80cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.754+0000 7f58ff393640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f815b700 con 0x7f58f80cf290 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.754+0000 7f58ff393640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f80cd9e0 con 0x7f58f806cf90 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.754+0000 7f58ff393640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8072120 con 0x7f58f80b4000 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd930640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58f806cf90 0x7f58f80cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd930640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58f806cf90 0x7f58f80cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38098/0 (socket says 172.21.15.105:38098) 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd930640 1 -- 172.21.15.105:0/3926396260 learned_addr learned my addr 172.21.15.105:0/3926396260 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd12f640 1 --2- 172.21.15.105:0/3926396260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80b4000 0x7f58f815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd930640 1 -- 172.21.15.105:0/3926396260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80b4000 msgr2=0x7f58f815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd930640 1 --2- 172.21.15.105:0/3926396260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80b4000 0x7f58f815b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd930640 1 -- 172.21.15.105:0/3926396260 >> v1:172.21.15.144:6789/0 conn(0x7f58f80cf290 legacy=0x7f58f80b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd930640 1 -- 172.21.15.105:0/3926396260 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f8072400 con 0x7f58f806cf90 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.755+0000 7f58fd930640 1 --2- 172.21.15.105:0/3926396260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f80cde70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009870 tx=0x7f58f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc5968feb2af085 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ebfff640 1 -- 172.21.15.105:0/3926396260 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4035020 con 0x7f58f806cf90 2024-03-08T08:27:56.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ebfff640 1 -- 172.21.15.105:0/3926396260 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f4031640 con 0x7f58f806cf90 2024-03-08T08:27:56.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ebfff640 1 -- 172.21.15.105:0/3926396260 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f402d040 con 0x7f58f806cf90 2024-03-08T08:27:56.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ff393640 1 -- 172.21.15.105:0/3926396260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 msgr2=0x7f58f80cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ff393640 1 --2- 172.21.15.105:0/3926396260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f80cde70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009870 tx=0x7f58f402b8f0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ff393640 1 -- 172.21.15.105:0/3926396260 shutdown_connections 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ff393640 1 --2- 172.21.15.105:0/3926396260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f80cde70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ff393640 1 --2- 172.21.15.105:0/3926396260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80b4000 0x7f58f815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ff393640 1 -- 172.21.15.105:0/3926396260 >> 172.21.15.105:0/3926396260 conn(0x7f58f80b1740 msgr2=0x7f58f80b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ff393640 1 -- 172.21.15.105:0/3926396260 shutdown_connections 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.756+0000 7f58ff393640 1 -- 172.21.15.105:0/3926396260 wait complete. 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.757+0000 7f58ff393640 1 Processor -- start 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.757+0000 7f58ff393640 1 -- start start 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58ff393640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58ff393640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f80b4000 0x7f58f806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58ff393640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80cf290 0x7f58f806dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58ff393640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f815a500 con 0x7f58f806cf90 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58ff393640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80726e0 con 0x7f58f80b4000 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58ff393640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f806d420 con 0x7f58f80cf290 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd12f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd12f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f814d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38102/0 (socket says 172.21.15.105:38102) 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd12f640 1 -- 172.21.15.105:0/172340226 learned_addr learned my addr 172.21.15.105:0/172340226 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd930640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80cf290 0x7f58f806dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fc92e640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f80b4000 0x7f58f806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd12f640 1 -- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80cf290 msgr2=0x7f58f806dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd12f640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80cf290 0x7f58f806dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd12f640 1 -- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f80b4000 msgr2=0x7f58f806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd12f640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f80b4000 0x7f58f806d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd12f640 1 -- 172.21.15.105:0/172340226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f81e3820 con 0x7f58f806cf90 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fc92e640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f80b4000 0x7f58f806d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:56.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.758+0000 7f58fd930640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80cf290 0x7f58f806dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:27:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.759+0000 7f58fd12f640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f814d2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c7f0 tx=0x7f58f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.759+0000 7f58e9ffb640 1 -- 172.21.15.105:0/172340226 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0015070 con 0x7f58f806cf90 2024-03-08T08:27:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.759+0000 7f58e9ffb640 1 -- 172.21.15.105:0/172340226 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0004240 con 0x7f58f806cf90 2024-03-08T08:27:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.759+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81e3b00 con 0x7f58f806cf90 2024-03-08T08:27:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.759+0000 7f58e9ffb640 1 -- 172.21.15.105:0/172340226 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0007620 con 0x7f58f806cf90 2024-03-08T08:27:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.759+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81e40c0 con 0x7f58f806cf90 2024-03-08T08:27:56.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.760+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f8075120 con 0x7f58f806cf90 2024-03-08T08:27:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.760+0000 7f58e9ffb640 1 -- 172.21.15.105:0/172340226 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58f00079d0 con 0x7f58f806cf90 2024-03-08T08:27:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.761+0000 7f58e9ffb640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f58dc06b4d0 0x7f58dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.761+0000 7f58e9ffb640 1 -- 172.21.15.105:0/172340226 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58f008a5f0 con 0x7f58f806cf90 2024-03-08T08:27:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.761+0000 7f58fc92e640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f58dc06b4d0 0x7f58dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.761+0000 7f58fc92e640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f58dc06b4d0 0x7f58dc06d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58f814fde0 tx=0x7f58ec009570 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:56.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.765+0000 7f58e9ffb640 1 -- 172.21.15.105:0/172340226 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58f0059500 con 0x7f58f806cf90 2024-03-08T08:27:56.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.970+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58f80b1bc0 con 0x7f58f806cf90 2024-03-08T08:27:56.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.970+0000 7f58e9ffb640 1 -- 172.21.15.105:0/172340226 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f58f0058e00 con 0x7f58f806cf90 2024-03-08T08:27:56.971 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f58dc06b4d0 msgr2=0x7f58dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f58dc06b4d0 0x7f58dc06d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58f814fde0 tx=0x7f58ec009570 comp rx=0 tx=0).stop 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 msgr2=0x7f58f814d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f814d2c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c7f0 tx=0x7f58f000ccb0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 shutdown_connections 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f58dc06b4d0 0x7f58dc06d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58f80cf290 0x7f58f806dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f80b4000 0x7f58f806d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 --2- 172.21.15.105:0/172340226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f806cf90 0x7f58f814d2c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 >> 172.21.15.105:0/172340226 conn(0x7f58f80b1740 msgr2=0x7f58f81515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 shutdown_connections 2024-03-08T08:27:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:56.972+0000 7f58ff393640 1 -- 172.21.15.105:0/172340226 wait complete. 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T08:27:56.984 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T08:27:56.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:56.985 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-03-08T08:27:56.999 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:27:56.999 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:27:56.999 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T08:27:57.000 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:57.001 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T08:27:57.001 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42706sec preferred_lft 42706sec 2024-03-08T08:27:57.001 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-08T08:27:57.001 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:57.001 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T08:27:57.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.001 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:57.001 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:57.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-03-08T08:27:57.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-03-08T08:27:57.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-03-08T08:27:57.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T08:27:57.002 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:57.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.171 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T08:27:57.197 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-03-08T08:27:57.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.199 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-03-08T08:27:57.269 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-03-08T08:27:57.269 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T08:27:57.269 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T08:27:57.269 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T08:27:57.269 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T08:27:57.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.270 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:57.270 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:57.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T08:27:57.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T08:27:57.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T08:27:57.270 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:57.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.530 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-08T08:27:57.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.545 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-03-08T08:27:57.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.599 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:57.599 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:57.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T08:27:57.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-08T08:27:57.600 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:57.682 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-08T08:27:57.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.683 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:57.683 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:57.683 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-08T08:27:57.683 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T08:27:57.683 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-08T08:27:57.683 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-08T08:27:57.683 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-08T08:27:57.683 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:57.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.830 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:57.830 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:57.830 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-03-08T08:27:57.830 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T08:27:57.831 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:57.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:27:57.872 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-08T08:27:57.927 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-08T08:27:57.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:27:57.927 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-08T08:27:57.982 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-08T08:27:57.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:57.982 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:27:58.067 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:27:58.067 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:27:58.222 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:27:58.222 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:27:58.222 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.144:3300,172.21.15.144:3301" 2024-03-08T08:27:58.222 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 0@1afff3d7-46c3-4cd2-9212-c89ee069e495.cephfs=/ 2024-03-08T08:27:58.222 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.105:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2024-03-08T08:27:58.223 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-08T08:27:58.223 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-03-08T08:27:58.223 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T08:27:58.223 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T08:27:58.223 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T08:27:58.225 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T08:27:58.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:58.225 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-08T08:27:58.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:58.252 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:27:58.312 INFO:teuthology.orchestra.run.smithi105.stdout:client4683 2024-03-08T08:27:58.312 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:58.312 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:27:58.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:58.367 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:27:58.428 INFO:teuthology.orchestra.run.smithi105.stdout:client4683 2024-03-08T08:27:58.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:58.428 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:27:58.488 INFO:teuthology.orchestra.run.smithi105.stdout:client4683 2024-03-08T08:27:58.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:58.488 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1afff3d7-46c3-4cd2-9212-c89ee069e495.client4683/status') 2024-03-08T08:27:58.558 INFO:teuthology.orchestra.run.smithi105.stdout:instance: client.4683 (3)192.168.0.1:0/2391627371 2024-03-08T08:27:58.558 INFO:teuthology.orchestra.run.smithi105.stdout:blocklisted: false 2024-03-08T08:27:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records in 2024-03-08T08:27:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records out 2024-03-08T08:27:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:69 bytes copied, 2.0207e-05 s, 3.4 MB/s 2024-03-08T08:27:58.559 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-08T08:27:58.560 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T08:27:58.560 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-08T08:27:58.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T08:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.699+0000 7fd037035640 1 Processor -- start 2024-03-08T08:27:58.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.699+0000 7fd037035640 1 -- start start 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd037035640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd03006e0a0 0x7fd03006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd037035640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03015b670 con 0x7fd0300b4d00 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd037035640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03015b940 con 0x7fd030072440 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd037035640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03006ea50 con 0x7fd03006e0a0 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd0355d2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd03006e0a0 0x7fd03006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd02ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd0300b4d00 0x7fd03015af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47604/0 (socket says 172.21.15.105:47604) 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd0355d2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd03006e0a0 0x7fd03006e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38106/0 (socket says 172.21.15.105:38106) 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd02ffff640 1 -- 172.21.15.105:0/1019540693 learned_addr learned my addr 172.21.15.105:0/1019540693 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd0355d2640 1 -- 172.21.15.105:0/1019540693 >> v1:172.21.15.144:6789/0 conn(0x7fd030072440 legacy=0x7fd030071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:27:58.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd0355d2640 1 -- 172.21.15.105:0/1019540693 >> v1:172.21.15.105:6789/0 conn(0x7fd0300b4d00 legacy=0x7fd03015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd0355d2640 1 -- 172.21.15.105:0/1019540693 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03015bc20 con 0x7fd03006e0a0 2024-03-08T08:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.700+0000 7fd02f7fe640 1 -- 172.21.15.105:0/1019540693 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057979077 0 0) 0x7fd03015b670 con 0x7fd0300b4d00 2024-03-08T08:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd0355d2640 1 --2- 172.21.15.105:0/1019540693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03006e0a0 0x7fd03006e510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd028009d90 tx=0x7fd02802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8057fdde21c08b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd02f7fe640 1 -- 172.21.15.105:0/1019540693 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028039070 con 0x7fd03006e0a0 2024-03-08T08:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd02f7fe640 1 -- 172.21.15.105:0/1019540693 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02802cc60 con 0x7fd03006e0a0 2024-03-08T08:27:58.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd02f7fe640 1 -- 172.21.15.105:0/1019540693 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028034660 con 0x7fd03006e0a0 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd037035640 1 -- 172.21.15.105:0/1019540693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03006e0a0 msgr2=0x7fd03006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd037035640 1 --2- 172.21.15.105:0/1019540693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03006e0a0 0x7fd03006e510 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd028009d90 tx=0x7fd02802beb0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd037035640 1 -- 172.21.15.105:0/1019540693 shutdown_connections 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd037035640 1 --2- 172.21.15.105:0/1019540693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03006e0a0 0x7fd03006e510 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd037035640 1 -- 172.21.15.105:0/1019540693 >> 172.21.15.105:0/1019540693 conn(0x7fd0300b1740 msgr2=0x7fd0300b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd037035640 1 -- 172.21.15.105:0/1019540693 shutdown_connections 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.701+0000 7fd037035640 1 -- 172.21.15.105:0/1019540693 wait complete. 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.703+0000 7fd037035640 1 Processor -- start 2024-03-08T08:27:58.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.703+0000 7fd037035640 1 -- start start 2024-03-08T08:27:58.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.703+0000 7fd037035640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd03006e0a0 0x7fd0301549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:58.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.703+0000 7fd037035640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd030072440 0x7fd030147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:58.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.703+0000 7fd037035640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0300b4d00 0x7fd030156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:58.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd037035640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd030072820 con 0x7fd030072440 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd037035640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd030070b20 con 0x7fd03006e0a0 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd037035640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300729a0 con 0x7fd0300b4d00 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd030072440 0x7fd030147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd030072440 0x7fd030147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38114/0 (socket says 172.21.15.105:38114) 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 -- 172.21.15.105:0/53181492 learned_addr learned my addr 172.21.15.105:0/53181492 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd034dd1640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd03006e0a0 0x7fd0301549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd0355d2640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0300b4d00 0x7fd030156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 -- 172.21.15.105:0/53181492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0300b4d00 msgr2=0x7fd030156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0300b4d00 0x7fd030156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 -- 172.21.15.105:0/53181492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd03006e0a0 msgr2=0x7fd0301549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd03006e0a0 0x7fd0301549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 -- 172.21.15.105:0/53181492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030155590 con 0x7fd030072440 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02ffff640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd030072440 0x7fd030147100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c410 tx=0x7fd01c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02d7fa640 1 -- 172.21.15.105:0/53181492 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0074e0 con 0x7fd030072440 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02d7fa640 1 -- 172.21.15.105:0/53181492 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c0047e0 con 0x7fd030072440 2024-03-08T08:27:58.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd030155870 con 0x7fd030072440 2024-03-08T08:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd02d7fa640 1 -- 172.21.15.105:0/53181492 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c004bd0 con 0x7fd030072440 2024-03-08T08:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.704+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd030155e30 con 0x7fd030072440 2024-03-08T08:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.705+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd030075120 con 0x7fd030072440 2024-03-08T08:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.706+0000 7fd02d7fa640 1 -- 172.21.15.105:0/53181492 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd01c011070 con 0x7fd030072440 2024-03-08T08:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.706+0000 7fd02d7fa640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd01406b4d0 0x7fd01406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:27:58.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.706+0000 7fd02d7fa640 1 -- 172.21.15.105:0/53181492 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd01c089e80 con 0x7fd030072440 2024-03-08T08:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.706+0000 7fd034dd1640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd01406b4d0 0x7fd01406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:27:58.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.706+0000 7fd034dd1640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd01406b4d0 0x7fd01406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd024007fd0 tx=0x7fd024009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:27:58.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.710+0000 7fd02d7fa640 1 -- 172.21.15.105:0/53181492 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd01c058cf0 con 0x7fd030072440 2024-03-08T08:27:58.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.918+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd0300cba20 con 0x7fd030072440 2024-03-08T08:27:58.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.918+0000 7fd02d7fa640 1 -- 172.21.15.105:0/53181492 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd01c0585f0 con 0x7fd030072440 2024-03-08T08:27:58.919 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T08:27:58.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd01406b4d0 msgr2=0x7fd01406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:58.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd01406b4d0 0x7fd01406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd024007fd0 tx=0x7fd024009040 comp rx=0 tx=0).stop 2024-03-08T08:27:58.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd030072440 msgr2=0x7fd030147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:27:58.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd030072440 0x7fd030147100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c410 tx=0x7fd01c00c8d0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 shutdown_connections 2024-03-08T08:27:58.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd01406b4d0 0x7fd01406d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0300b4d00 0x7fd030156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd030072440 0x7fd030147100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 --2- 172.21.15.105:0/53181492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd03006e0a0 0x7fd0301549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:27:58.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 >> 172.21.15.105:0/53181492 conn(0x7fd0300b1740 msgr2=0x7fd0300b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:27:58.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 shutdown_connections 2024-03-08T08:27:58.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:27:58.920+0000 7fd037035640 1 -- 172.21.15.105:0/53181492 wait complete. 2024-03-08T08:27:58.931 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T08:27:58.931 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T08:27:58.931 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-03-08T08:27:58.931 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-08T08:27:58.931 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-08T08:27:58.931 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T08:27:58.931 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T08:27:58.932 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T08:27:58.932 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-08T08:27:58.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:58.932 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-03-08T08:27:58.946 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:58.947 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42704sec preferred_lft 42704sec 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:68:08:f2:11:a9 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::20ed:97ff:fe3a:c855/64 scope link 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:68:08:f2:11:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::68:8ff:fef2:11a9/64 scope link tentative 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:58.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:58.949 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T08:27:59.012 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-03-08T08:27:59.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.013 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-03-08T08:27:59.069 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-03-08T08:27:59.069 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T08:27:59.069 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T08:27:59.069 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T08:27:59.069 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T08:27:59.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.070 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:59.070 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:59.070 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T08:27:59.070 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T08:27:59.070 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T08:27:59.070 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:59.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.165 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-08T08:27:59.179 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T08:27:59.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.179 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-03-08T08:27:59.233 INFO:teuthology.orchestra.run.smithi105.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-08T08:27:59.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.233 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:59.233 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:59.234 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T08:27:59.234 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-08T08:27:59.234 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:59.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.315 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-08T08:27:59.382 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:27:59.382 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:27:59.382 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:27:59.382 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:59.382 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-08T08:27:59.382 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:59.382 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T08:27:59.383 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 3a:27:67:49:95:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T08:27:59.383 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T08:27:59.383 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:59.383 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::3827:67ff:fe49:9505/64 scope link 2024-03-08T08:27:59.383 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:59.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.384 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-08T08:27:59.450 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:27:59.450 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:27:59.450 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:27:59.450 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:59.451 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-08T08:27:59.451 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:59.451 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T08:27:59.451 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 3a:27:67:49:95:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T08:27:59.451 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T08:27:59.451 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:59.451 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::3827:67ff:fe49:9505/64 scope link 2024-03-08T08:27:59.451 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:27:59.452 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-08T08:27:59.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.452 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:59.452 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:59.452 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-08T08:27:59.452 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-08T08:27:59.452 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-08T08:27:59.452 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-08T08:27:59.452 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-08T08:27:59.453 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:59.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.575 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:27:59.575 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:27:59.575 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.1 up 2024-03-08T08:27:59.575 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.1 master ceph-brx 2024-03-08T08:27:59.575 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:27:59.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:27:59.616 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-08T08:27:59.671 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-08T08:27:59.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:27:59.672 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-08T08:27:59.726 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-08T08:27:59.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.726 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:27:59.811 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:27:59.811 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:27:59.821 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:27:59.821 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:27:59.821 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.144:3300,172.21.15.144:3301" 2024-03-08T08:27:59.821 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 2@1afff3d7-46c3-4cd2-9212-c89ee069e495.cephfs=/ 2024-03-08T08:27:59.821 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.105:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2024-03-08T08:27:59.821 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-08T08:27:59.821 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-03-08T08:27:59.822 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T08:27:59.822 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T08:27:59.822 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T08:27:59.823 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T08:27:59.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-08T08:27:59.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.849 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:27:59.906 INFO:teuthology.orchestra.run.smithi105.stdout:client4695 2024-03-08T08:27:59.906 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:27:59.906 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:27:59.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:27:59.961 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:00.017 INFO:teuthology.orchestra.run.smithi105.stdout:client4695 2024-03-08T08:28:00.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:00.017 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:00.074 INFO:teuthology.orchestra.run.smithi105.stdout:client4695 2024-03-08T08:28:00.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:00.074 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1afff3d7-46c3-4cd2-9212-c89ee069e495.client4695/status') 2024-03-08T08:28:00.140 INFO:teuthology.orchestra.run.smithi105.stdout:instance: client.4695 (3)192.168.0.2:0/3265033867 2024-03-08T08:28:00.140 INFO:teuthology.orchestra.run.smithi105.stdout:blocklisted: false 2024-03-08T08:28:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records in 2024-03-08T08:28:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records out 2024-03-08T08:28:00.140 INFO:teuthology.orchestra.run.smithi105.stderr:69 bytes copied, 3.6398e-05 s, 1.9 MB/s 2024-03-08T08:28:00.141 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-08T08:28:00.141 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T08:28:00.141 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:00.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T08:28:00.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.280+0000 7f60fba07640 1 Processor -- start 2024-03-08T08:28:00.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.280+0000 7f60fba07640 1 -- start start 2024-03-08T08:28:00.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60fba07640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f406b390 0x7f60f406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60fba07640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f60f40b5990 0x7f60f40ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60fba07640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f406f830 con 0x7f60f415df20 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60fba07640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f406dbb0 con 0x7f60f40b5990 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60fba07640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f406fad0 con 0x7f60f406b390 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60f9fa4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f60f40b5990 0x7f60f40ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60f97a3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f60f415df20 0x7f60f406ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47606/0 (socket says 172.21.15.105:47606) 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60f97a3640 1 -- 172.21.15.105:0/2370250245 learned_addr learned my addr 172.21.15.105:0/2370250245 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.281+0000 7f60f8fa2640 1 --2- 172.21.15.105:0/2370250245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f406b390 0x7f60f406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60f9fa4640 1 -- 172.21.15.105:0/2370250245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f406b390 msgr2=0x7f60f406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60f9fa4640 1 --2- 172.21.15.105:0/2370250245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f406b390 0x7f60f406c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60f9fa4640 1 -- 172.21.15.105:0/2370250245 >> v1:172.21.15.105:6789/0 conn(0x7f60f415df20 legacy=0x7f60f406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60ebfff640 1 -- 172.21.15.105:0/2370250245 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495359233 0 0) 0x7f60f406f830 con 0x7f60f415df20 2024-03-08T08:28:00.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60f9fa4640 1 -- 172.21.15.105:0/2370250245 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f406fdb0 con 0x7f60f40b5990 2024-03-08T08:28:00.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60f9fa4640 1 --2- 172.21.15.105:0/2370250245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f40b5990 0x7f60f40ad970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0097a0 tx=0x7f60ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360b1e94cfddd0ce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60ebfff640 1 -- 172.21.15.105:0/2370250245 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0315a0 con 0x7f60f40b5990 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60ebfff640 1 -- 172.21.15.105:0/2370250245 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec037070 con 0x7f60f40b5990 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.282+0000 7f60ebfff640 1 -- 172.21.15.105:0/2370250245 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec031960 con 0x7f60f40b5990 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.283+0000 7f60fba07640 1 -- 172.21.15.105:0/2370250245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f40b5990 msgr2=0x7f60f40ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.283+0000 7f60fba07640 1 --2- 172.21.15.105:0/2370250245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f40b5990 0x7f60f40ad970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0097a0 tx=0x7f60ec02b840 comp rx=0 tx=0).stop 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.283+0000 7f60fba07640 1 -- 172.21.15.105:0/2370250245 shutdown_connections 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.283+0000 7f60fba07640 1 --2- 172.21.15.105:0/2370250245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f40b5990 0x7f60f40ad970 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.283+0000 7f60fba07640 1 --2- 172.21.15.105:0/2370250245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f406b390 0x7f60f406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.283+0000 7f60fba07640 1 -- 172.21.15.105:0/2370250245 >> 172.21.15.105:0/2370250245 conn(0x7f60f406bd90 msgr2=0x7f60f406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.283+0000 7f60fba07640 1 -- 172.21.15.105:0/2370250245 shutdown_connections 2024-03-08T08:28:00.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.283+0000 7f60fba07640 1 -- 172.21.15.105:0/2370250245 wait complete. 2024-03-08T08:28:00.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.284+0000 7f60fba07640 1 Processor -- start 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.284+0000 7f60fba07640 1 -- start start 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60fba07640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60f406b390 0x7f60f40cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60fba07640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f40b5990 0x7f60f41f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60fba07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f415df20 0x7f60f40aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60fba07640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40adeb0 con 0x7f60f415df20 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60fba07640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f406e0e0 con 0x7f60f406b390 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60fba07640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40ae030 con 0x7f60f40b5990 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f9fa4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f415df20 0x7f60f40aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f9fa4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f415df20 0x7f60f40aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38144/0 (socket says 172.21.15.105:38144) 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f9fa4640 1 -- 172.21.15.105:0/3221286093 learned_addr learned my addr 172.21.15.105:0/3221286093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f8fa2640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f40b5990 0x7f60f41f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f97a3640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60f406b390 0x7f60f40cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f9fa4640 1 -- 172.21.15.105:0/3221286093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f40b5990 msgr2=0x7f60f41f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f9fa4640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f40b5990 0x7f60f41f3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f9fa4640 1 -- 172.21.15.105:0/3221286093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60f406b390 msgr2=0x7f60f40cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:00.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f9fa4640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60f406b390 0x7f60f40cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.285+0000 7f60f9fa4640 1 -- 172.21.15.105:0/3221286093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f41f3b20 con 0x7f60f415df20 2024-03-08T08:28:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.286+0000 7f60f9fa4640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f415df20 0x7f60f40aebc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0098c0 tx=0x7f60ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.286+0000 7f60e9ffb640 1 -- 172.21.15.105:0/3221286093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec032630 con 0x7f60f415df20 2024-03-08T08:28:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.286+0000 7f60e9ffb640 1 -- 172.21.15.105:0/3221286093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec0327d0 con 0x7f60f415df20 2024-03-08T08:28:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.286+0000 7f60e9ffb640 1 -- 172.21.15.105:0/3221286093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec032bc0 con 0x7f60f415df20 2024-03-08T08:28:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.286+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f41f3e00 con 0x7f60f415df20 2024-03-08T08:28:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.286+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f415a250 con 0x7f60f415df20 2024-03-08T08:28:00.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.287+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f415a450 con 0x7f60f415df20 2024-03-08T08:28:00.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.287+0000 7f60e9ffb640 1 -- 172.21.15.105:0/3221286093 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60ec04c020 con 0x7f60f415df20 2024-03-08T08:28:00.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.288+0000 7f60e9ffb640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f60d806b4d0 0x7f60d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:00.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.288+0000 7f60e9ffb640 1 -- 172.21.15.105:0/3221286093 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60ec02f020 con 0x7f60f415df20 2024-03-08T08:28:00.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.288+0000 7f60f97a3640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f60d806b4d0 0x7f60d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:00.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.288+0000 7f60f97a3640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f60d806b4d0 0x7f60d806d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f60f00012e0 tx=0x7f60f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:00.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.292+0000 7f60e9ffb640 1 -- 172.21.15.105:0/3221286093 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60f415a450 con 0x7f60f415df20 2024-03-08T08:28:00.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.501+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f60f4075120 con 0x7f60f415df20 2024-03-08T08:28:00.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.502+0000 7f60e9ffb640 1 -- 172.21.15.105:0/3221286093 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f60ec040070 con 0x7f60f415df20 2024-03-08T08:28:00.503 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T08:28:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f60d806b4d0 msgr2=0x7f60d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f60d806b4d0 0x7f60d806d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f60f00012e0 tx=0x7f60f0009040 comp rx=0 tx=0).stop 2024-03-08T08:28:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f415df20 msgr2=0x7f60f40aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f415df20 0x7f60f40aebc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0098c0 tx=0x7f60ec0046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 shutdown_connections 2024-03-08T08:28:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f60d806b4d0 0x7f60d806d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60f415df20 0x7f60f40aebc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60f40b5990 0x7f60f41f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 --2- 172.21.15.105:0/3221286093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60f406b390 0x7f60f40cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 >> 172.21.15.105:0/3221286093 conn(0x7f60f406bd90 msgr2=0x7f60f406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 shutdown_connections 2024-03-08T08:28:00.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:00.504+0000 7f60fba07640 1 -- 172.21.15.105:0/3221286093 wait complete. 2024-03-08T08:28:00.515 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T08:28:00.515 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T08:28:00.515 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-03-08T08:28:00.515 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-08T08:28:00.515 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:00.515 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T08:28:00.515 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T08:28:00.516 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T08:28:00.516 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T08:28:00.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:00.516 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-03-08T08:28:00.534 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:00.535 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T08:28:00.536 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:00.536 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T08:28:00.536 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42638sec preferred_lft 42638sec 2024-03-08T08:28:00.536 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-03-08T08:28:00.536 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:00.536 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T08:28:00.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:00.536 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:00.536 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:00.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add name ceph-brx type bridge 2024-03-08T08:28:00.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr flush dev ceph-brx 2024-03-08T08:28:00.537 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set ceph-brx up 2024-03-08T08:28:00.537 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T08:28:00.537 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:00.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:00.716 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T08:28:00.744 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-03-08T08:28:00.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:00.746 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-03-08T08:28:00.826 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-03-08T08:28:00.826 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T08:28:00.827 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T08:28:00.827 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T08:28:00.827 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T08:28:00.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:00.827 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:00.827 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:00.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T08:28:00.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T08:28:00.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T08:28:00.827 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:01.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.084 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-03-08T08:28:01.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.099 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2024-03-08T08:28:01.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.154 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:01.154 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:01.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T08:28:01.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-08T08:28:01.154 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:01.236 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-08T08:28:01.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.236 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:01.236 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:01.237 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-08T08:28:01.237 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T08:28:01.237 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-08T08:28:01.237 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-08T08:28:01.237 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-08T08:28:01.237 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:01.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.370 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:01.370 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:01.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2024-03-08T08:28:01.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T08:28:01.371 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:01.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:28:01.412 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:01.467 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-08T08:28:01.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:28:01.467 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:01.521 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-08T08:28:01.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.521 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:01.606 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:01.606 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:28:01.718 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:01.718 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:28:01.718 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.144:3300,172.21.15.144:3301" 2024-03-08T08:28:01.718 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@1afff3d7-46c3-4cd2-9212-c89ee069e495.cephfs=/ 2024-03-08T08:28:01.718 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.105:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2024-03-08T08:28:01.719 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-08T08:28:01.719 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2024-03-08T08:28:01.719 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T08:28:01.719 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T08:28:01.719 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T08:28:01.720 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T08:28:01.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.721 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:01.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.788 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:01.844 INFO:teuthology.orchestra.run.smithi144.stdout:client4707 2024-03-08T08:28:01.844 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:01.844 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:01.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.899 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:01.956 INFO:teuthology.orchestra.run.smithi144.stdout:client4707 2024-03-08T08:28:01.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:01.956 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:02.011 INFO:teuthology.orchestra.run.smithi144.stdout:client4707 2024-03-08T08:28:02.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.012 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1afff3d7-46c3-4cd2-9212-c89ee069e495.client4707/status') 2024-03-08T08:28:02.078 INFO:teuthology.orchestra.run.smithi144.stdout:instance: client.4707 (3)172.21.15.144:0/4263822627 2024-03-08T08:28:02.078 INFO:teuthology.orchestra.run.smithi144.stdout:blocklisted: false 2024-03-08T08:28:02.079 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records in 2024-03-08T08:28:02.079 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records out 2024-03-08T08:28:02.079 INFO:teuthology.orchestra.run.smithi144.stderr:71 bytes copied, 2.0226e-05 s, 3.5 MB/s 2024-03-08T08:28:02.080 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-08T08:28:02.080 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T08:28:02.080 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-08T08:28:02.080 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T08:28:02.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.191+0000 7fdfa2f79640 1 Processor -- start 2024-03-08T08:28:02.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.191+0000 7fdfa2f79640 1 -- start start 2024-03-08T08:28:02.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa2f79640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdf9c1453d0 0x7fdf9c0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:02.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa2f79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 0x7fdf9c0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:02.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa2f79640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf9c0cd050 0x7fdf9c15f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:02.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa2f79640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9c1457b0 con 0x7fdf9c0cd050 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa2f79640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b44e0 con 0x7fdf9c1453d0 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa2f79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b47c0 con 0x7fdf9c0cf340 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa1516640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf9c0cd050 0x7fdf9c15f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa1516640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdf9c0cd050 0x7fdf9c15f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38150/0 (socket says 172.21.15.105:38150) 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa1516640 1 -- 172.21.15.105:0/1291215507 learned_addr learned my addr 172.21.15.105:0/1291215507 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdf93fff640 1 --2- 172.21.15.105:0/1291215507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 0x7fdf9c0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa0d15640 1 --2- 172.21.15.105:0/1291215507 >> v2:172.21.15.144:3300/0 conn(0x7fdf9c1453d0 0x7fdf9c0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa1516640 1 -- 172.21.15.105:0/1291215507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 msgr2=0x7fdf9c0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa1516640 1 --2- 172.21.15.105:0/1291215507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 0x7fdf9c0ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa1516640 1 -- 172.21.15.105:0/1291215507 >> v2:172.21.15.144:3300/0 conn(0x7fdf9c1453d0 msgr2=0x7fdf9c0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa1516640 1 --2- 172.21.15.105:0/1291215507 >> v2:172.21.15.144:3300/0 conn(0x7fdf9c1453d0 0x7fdf9c0cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.192+0000 7fdfa1516640 1 -- 172.21.15.105:0/1291215507 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c0b4aa0 con 0x7fdf9c0cd050 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.193+0000 7fdfa1516640 1 --2- 172.21.15.105:0/1291215507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c0cd050 0x7fdf9c15f630 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c06bd80 tx=0x7fdf9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f150b76c0f396ef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.193+0000 7fdf937fe640 1 -- 172.21.15.105:0/1291215507 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9402ca70 con 0x7fdf9c0cd050 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.193+0000 7fdf937fe640 1 -- 172.21.15.105:0/1291215507 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9402cc10 con 0x7fdf9c0cd050 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.193+0000 7fdf937fe640 1 -- 172.21.15.105:0/1291215507 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf940343f0 con 0x7fdf9c0cd050 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1291215507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c0cd050 msgr2=0x7fdf9c15f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1291215507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c0cd050 0x7fdf9c15f630 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c06bd80 tx=0x7fdf9402bed0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1291215507 shutdown_connections 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1291215507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c0cd050 0x7fdf9c15f630 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1291215507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 0x7fdf9c0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1291215507 >> v2:172.21.15.144:3300/0 conn(0x7fdf9c1453d0 0x7fdf9c0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1291215507 >> 172.21.15.105:0/1291215507 conn(0x7fdf9c0b1740 msgr2=0x7fdf9c0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:02.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1291215507 shutdown_connections 2024-03-08T08:28:02.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.194+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1291215507 wait complete. 2024-03-08T08:28:02.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.195+0000 7fdfa2f79640 1 Processor -- start 2024-03-08T08:28:02.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.195+0000 7fdfa2f79640 1 -- start start 2024-03-08T08:28:02.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa2f79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9c0cd050 0x7fdf9c13d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:02.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa2f79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 0x7fdf9c15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:02.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa2f79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c1453d0 0x7fdf9c140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:02.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa2f79640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c15fb70 con 0x7fdf9c1453d0 2024-03-08T08:28:02.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa2f79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b4190 con 0x7fdf9c0cd050 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa2f79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c15fcf0 con 0x7fdf9c0cf340 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c1453d0 0x7fdf9c140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c1453d0 0x7fdf9c140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38156/0 (socket says 172.21.15.105:38156) 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 -- 172.21.15.105:0/1837984770 learned_addr learned my addr 172.21.15.105:0/1837984770 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdf93fff640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 0x7fdf9c15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa0d15640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9c0cd050 0x7fdf9c13d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 -- 172.21.15.105:0/1837984770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 msgr2=0x7fdf9c15b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 0x7fdf9c15b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 -- 172.21.15.105:0/1837984770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9c0cd050 msgr2=0x7fdf9c13d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9c0cd050 0x7fdf9c13d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 -- 172.21.15.105:0/1837984770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c140de0 con 0x7fdf9c1453d0 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.196+0000 7fdfa1516640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c1453d0 0x7fdf9c140520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009f40 tx=0x7fdf94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.197+0000 7fdf917fa640 1 -- 172.21.15.105:0/1837984770 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94043020 con 0x7fdf9c1453d0 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.197+0000 7fdf917fa640 1 -- 172.21.15.105:0/1837984770 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9403e450 con 0x7fdf9c1453d0 2024-03-08T08:28:02.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.197+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c1e58e0 con 0x7fdf9c1453d0 2024-03-08T08:28:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.197+0000 7fdf917fa640 1 -- 172.21.15.105:0/1837984770 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94009510 con 0x7fdf9c1453d0 2024-03-08T08:28:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.197+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c1e5ea0 con 0x7fdf9c1453d0 2024-03-08T08:28:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.198+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c1e5700 con 0x7fdf9c1453d0 2024-03-08T08:28:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.198+0000 7fdf917fa640 1 -- 172.21.15.105:0/1837984770 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf94046020 con 0x7fdf9c1453d0 2024-03-08T08:28:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.198+0000 7fdf917fa640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.198+0000 7fdf917fa640 1 -- 172.21.15.105:0/1837984770 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf940ac770 con 0x7fdf9c1453d0 2024-03-08T08:28:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.198+0000 7fdfa0d15640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:02.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.199+0000 7fdfa0d15640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf8806b4d0 0x7fdf8806d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009aa0 tx=0x7fdf98009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:02.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.203+0000 7fdf917fa640 1 -- 172.21.15.105:0/1837984770 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf9c1e5700 con 0x7fdf9c1453d0 2024-03-08T08:28:02.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.411+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf9c075120 con 0x7fdf9c1453d0 2024-03-08T08:28:02.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.411+0000 7fdf917fa640 1 -- 172.21.15.105:0/1837984770 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdf9404d030 con 0x7fdf9c1453d0 2024-03-08T08:28:02.412 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T08:28:02.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf8806b4d0 msgr2=0x7fdf8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:02.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf8806b4d0 0x7fdf8806d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009aa0 tx=0x7fdf98009190 comp rx=0 tx=0).stop 2024-03-08T08:28:02.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c1453d0 msgr2=0x7fdf9c140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:02.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c1453d0 0x7fdf9c140520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009f40 tx=0x7fdf94004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:02.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 shutdown_connections 2024-03-08T08:28:02.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf9c1453d0 0x7fdf9c140520 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf9c0cf340 0x7fdf9c15b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 --2- 172.21.15.105:0/1837984770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf9c0cd050 0x7fdf9c13d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:02.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 >> 172.21.15.105:0/1837984770 conn(0x7fdf9c0b1740 msgr2=0x7fdf9c0aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:02.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 shutdown_connections 2024-03-08T08:28:02.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:02.413+0000 7fdfa2f79640 1 -- 172.21.15.105:0/1837984770 wait complete. 2024-03-08T08:28:02.424 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T08:28:02.424 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T08:28:02.424 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-03-08T08:28:02.425 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-08T08:28:02.425 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-08T08:28:02.425 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T08:28:02.425 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T08:28:02.425 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T08:28:02.425 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-08T08:28:02.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.425 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:02.441 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42636sec preferred_lft 42636sec 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether d6:2a:4f:6b:63:46 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::dc81:beff:feca:aa78/64 scope link 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether d6:2a:4f:6b:63:46 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T08:28:02.442 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::d42a:4fff:fe6b:6346/64 scope link tentative 2024-03-08T08:28:02.443 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.443 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T08:28:02.507 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-03-08T08:28:02.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.508 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-03-08T08:28:02.563 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-03-08T08:28:02.563 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T08:28:02.564 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T08:28:02.564 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T08:28:02.564 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T08:28:02.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.564 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:02.564 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:02.564 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T08:28:02.564 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T08:28:02.564 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T08:28:02.564 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:02.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.660 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-03-08T08:28:02.674 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T08:28:02.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.674 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2024-03-08T08:28:02.728 INFO:teuthology.orchestra.run.smithi144.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-08T08:28:02.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.729 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:02.729 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:02.729 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T08:28:02.729 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-08T08:28:02.729 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:02.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.811 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-08T08:28:02.877 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:28:02.877 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:28:02.877 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:28:02.877 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.877 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-03-08T08:28:02.877 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.877 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T08:28:02.877 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 1e:67:1b:e5:15:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T08:28:02.878 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T08:28:02.878 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.878 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1c67:1bff:fee5:15c5/64 scope link tentative 2024-03-08T08:28:02.878 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.879 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 1e:67:1b:e5:15:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-08T08:28:02.945 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1c67:1bff:fee5:15c5/64 scope link tentative 2024-03-08T08:28:02.946 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:02.946 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-08T08:28:02.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:02.947 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:02.947 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:02.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-08T08:28:02.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-08T08:28:02.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-08T08:28:02.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-08T08:28:02.947 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-08T08:28:02.947 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:03.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:03.071 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:03.071 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:03.072 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.1 up 2024-03-08T08:28:03.072 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.1 master ceph-brx 2024-03-08T08:28:03.072 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:03.112 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:28:03.112 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-08T08:28:03.168 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-08T08:28:03.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:28:03.168 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-08T08:28:03.222 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-08T08:28:03.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:03.222 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:03.306 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:03.306 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:28:03.314 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:03.314 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:28:03.314 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.144:3300,172.21.15.144:3301" 2024-03-08T08:28:03.314 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 3@1afff3d7-46c3-4cd2-9212-c89ee069e495.cephfs=/ 2024-03-08T08:28:03.314 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.105:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2024-03-08T08:28:03.314 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-08T08:28:03.315 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2024-03-08T08:28:03.315 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T08:28:03.315 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T08:28:03.315 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T08:28:03.316 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T08:28:03.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:03.316 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-08T08:28:03.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:03.342 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:03.399 INFO:teuthology.orchestra.run.smithi144.stdout:client4627 2024-03-08T08:28:03.399 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:03.399 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:03.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:03.454 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:03.510 INFO:teuthology.orchestra.run.smithi144.stdout:client4627 2024-03-08T08:28:03.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:03.511 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:03.566 INFO:teuthology.orchestra.run.smithi144.stdout:client4627 2024-03-08T08:28:03.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:03.567 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1afff3d7-46c3-4cd2-9212-c89ee069e495.client4627/status') 2024-03-08T08:28:03.633 INFO:teuthology.orchestra.run.smithi144.stdout:instance: client.4627 (3)192.168.0.2:0/2248554592 2024-03-08T08:28:03.633 INFO:teuthology.orchestra.run.smithi144.stdout:blocklisted: false 2024-03-08T08:28:03.633 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records in 2024-03-08T08:28:03.633 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records out 2024-03-08T08:28:03.633 INFO:teuthology.orchestra.run.smithi144.stderr:69 bytes copied, 1.9792e-05 s, 3.5 MB/s 2024-03-08T08:28:03.635 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-08T08:28:03.650 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.761+0000 7f2140039640 1 Processor -- start 2024-03-08T08:28:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.762+0000 7f2140039640 1 -- start start 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.762+0000 7f2140039640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f213815aad0 0x7f213815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.762+0000 7f2140039640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213806edf0 0x7f213815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.762+0000 7f2140039640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213815b360 con 0x7f213815aeb0 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.762+0000 7f2140039640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213815ebb0 con 0x7f213815aad0 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.762+0000 7f2140039640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213815ee90 con 0x7f213806edf0 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.763+0000 7f213ddd5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f213815aad0 0x7f213815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.763+0000 7f213ddd5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f213815aad0 0x7f213815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59032/0 (socket says 172.21.15.105:59032) 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.763+0000 7f213ddd5640 1 -- 172.21.15.105:0/1716584957 learned_addr learned my addr 172.21.15.105:0/1716584957 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:03.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.763+0000 7f213e5d6640 1 --2- 172.21.15.105:0/1716584957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213806edf0 0x7f213815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:03.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.763+0000 7f213ddd5640 1 -- 172.21.15.105:0/1716584957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213806edf0 msgr2=0x7f213815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:03.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.763+0000 7f213ddd5640 1 --2- 172.21.15.105:0/1716584957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213806edf0 0x7f213815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.763+0000 7f213ddd5640 1 -- 172.21.15.105:0/1716584957 >> v1:172.21.15.144:6789/0 conn(0x7f213815aeb0 legacy=0x7f213806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:03.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.763+0000 7f213ddd5640 1 -- 172.21.15.105:0/1716584957 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213815f170 con 0x7f213815aad0 2024-03-08T08:28:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f213ddd5640 1 --2- 172.21.15.105:0/1716584957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213815aad0 0x7f213815e5d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f213000d150 tx=0x7f213002e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaa5c8a429bb7ab server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f213cdd3640 1 -- 172.21.15.105:0/1716584957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000b0d0 con 0x7f213815aad0 2024-03-08T08:28:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f213cdd3640 1 -- 172.21.15.105:0/1716584957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213000b270 con 0x7f213815aad0 2024-03-08T08:28:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f213cdd3640 1 -- 172.21.15.105:0/1716584957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130009600 con 0x7f213815aad0 2024-03-08T08:28:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f2140039640 1 -- 172.21.15.105:0/1716584957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213815aad0 msgr2=0x7f213815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f2140039640 1 --2- 172.21.15.105:0/1716584957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213815aad0 0x7f213815e5d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f213000d150 tx=0x7f213002e160 comp rx=0 tx=0).stop 2024-03-08T08:28:03.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f2140039640 1 -- 172.21.15.105:0/1716584957 shutdown_connections 2024-03-08T08:28:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f2140039640 1 --2- 172.21.15.105:0/1716584957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213806edf0 0x7f213815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f2140039640 1 --2- 172.21.15.105:0/1716584957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213815aad0 0x7f213815e5d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.764+0000 7f2140039640 1 -- 172.21.15.105:0/1716584957 >> 172.21.15.105:0/1716584957 conn(0x7f213806bd90 msgr2=0x7f213806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.765+0000 7f2140039640 1 -- 172.21.15.105:0/1716584957 shutdown_connections 2024-03-08T08:28:03.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.765+0000 7f2140039640 1 -- 172.21.15.105:0/1716584957 wait complete. 2024-03-08T08:28:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f2140039640 1 Processor -- start 2024-03-08T08:28:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f2140039640 1 -- start start 2024-03-08T08:28:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f2140039640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213806edf0 0x7f21380cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f2140039640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f213815aad0 0x7f21380cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f2140039640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213815aeb0 0x7f2138154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f2140039640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380728a0 con 0x7f213806edf0 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f2140039640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213815c240 con 0x7f213815aad0 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f2140039640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213815cca0 con 0x7f213815aeb0 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.766+0000 7f213ddd5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213806edf0 0x7f21380cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213ddd5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213806edf0 0x7f21380cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59046/0 (socket says 172.21.15.105:59046) 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213ddd5640 1 -- 172.21.15.105:0/472220936 learned_addr learned my addr 172.21.15.105:0/472220936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213e5d6640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213815aeb0 0x7f2138154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213d5d4640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f213815aad0 0x7f21380cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213ddd5640 1 -- 172.21.15.105:0/472220936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213815aeb0 msgr2=0x7f2138154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213ddd5640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213815aeb0 0x7f2138154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213ddd5640 1 -- 172.21.15.105:0/472220936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f213815aad0 msgr2=0x7f21380cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213ddd5640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f213815aad0 0x7f21380cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213ddd5640 1 -- 172.21.15.105:0/472220936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380ce810 con 0x7f213806edf0 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f213ddd5640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213806edf0 0x7f21380cf8b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2130038040 tx=0x7f2130004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f212e7fc640 1 -- 172.21.15.105:0/472220936 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000b0d0 con 0x7f213806edf0 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f212e7fc640 1 -- 172.21.15.105:0/472220936 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2130045070 con 0x7f213806edf0 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21380ceaf0 con 0x7f213806edf0 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f212e7fc640 1 -- 172.21.15.105:0/472220936 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000b720 con 0x7f213806edf0 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.767+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21380cf0b0 con 0x7f213806edf0 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.768+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2138075120 con 0x7f213806edf0 2024-03-08T08:28:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.769+0000 7f212e7fc640 1 -- 172.21.15.105:0/472220936 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f213002ed10 con 0x7f213806edf0 2024-03-08T08:28:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.769+0000 7f212e7fc640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f211c06b4d0 0x7f211c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.769+0000 7f212e7fc640 1 -- 172.21.15.105:0/472220936 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21300accf0 con 0x7f213806edf0 2024-03-08T08:28:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.769+0000 7f213d5d4640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f211c06b4d0 0x7f211c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.769+0000 7f213d5d4640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f211c06b4d0 0x7f211c06d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2138159e00 tx=0x7f2128009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.773+0000 7f212e7fc640 1 -- 172.21.15.105:0/472220936 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21300b1070 con 0x7f213806edf0 2024-03-08T08:28:03.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.945+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21380cf4d0 con 0x7f213806edf0 2024-03-08T08:28:03.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.946+0000 7f212e7fc640 1 -- 172.21.15.105:0/472220936 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f213007b640 con 0x7f213806edf0 2024-03-08T08:28:03.947 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:03.948 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f211c06b4d0 msgr2=0x7f211c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:03.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f211c06b4d0 0x7f211c06d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2138159e00 tx=0x7f2128009040 comp rx=0 tx=0).stop 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213806edf0 msgr2=0x7f21380cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213806edf0 0x7f21380cf8b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2130038040 tx=0x7f2130004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 shutdown_connections 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f211c06b4d0 0x7f211c06d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f213815aeb0 0x7f2138154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f213815aad0 0x7f21380cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 --2- 172.21.15.105:0/472220936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f213806edf0 0x7f21380cf8b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 >> 172.21.15.105:0/472220936 conn(0x7f213806bd90 msgr2=0x7f213815de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 shutdown_connections 2024-03-08T08:28:03.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:03.948+0000 7f2140039640 1 -- 172.21.15.105:0/472220936 wait complete. 2024-03-08T08:28:03.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:03.961 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.062+0000 7f95a806f640 1 Processor -- start 2024-03-08T08:28:04.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.063+0000 7f95a806f640 1 -- start start 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.063+0000 7f95a806f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a015df20 0x7f95a006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.063+0000 7f95a806f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95a006b390 0x7f95a00ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.063+0000 7f95a806f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a006f830 con 0x7f95a00b5990 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.063+0000 7f95a806f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a006dbb0 con 0x7f95a006b390 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.063+0000 7f95a806f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a006fad0 con 0x7f95a015df20 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.063+0000 7f95a660c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95a006b390 0x7f95a00ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.063+0000 7f95a660c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95a006b390 0x7f95a00ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59060/0 (socket says 172.21.15.105:59060) 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.064+0000 7f95a660c640 1 -- 172.21.15.105:0/1005881318 learned_addr learned my addr 172.21.15.105:0/1005881318 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.064+0000 7f95a5e0b640 1 --2- 172.21.15.105:0/1005881318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a015df20 0x7f95a006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.064+0000 7f95a660c640 1 -- 172.21.15.105:0/1005881318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a015df20 msgr2=0x7f95a006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.064+0000 7f95a660c640 1 --2- 172.21.15.105:0/1005881318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a015df20 0x7f95a006c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.064+0000 7f95a660c640 1 -- 172.21.15.105:0/1005881318 >> v1:172.21.15.144:6789/0 conn(0x7f95a00b5990 legacy=0x7f95a006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.064+0000 7f95a660c640 1 -- 172.21.15.105:0/1005881318 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a006fdb0 con 0x7f95a006b390 2024-03-08T08:28:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.064+0000 7f95a5e0b640 1 --2- 172.21.15.105:0/1005881318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a015df20 0x7f95a006c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a660c640 1 --2- 172.21.15.105:0/1005881318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a006b390 0x7f95a00ad970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9598009870 tx=0x7f959802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ddc096c1a9b75d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a4e09640 1 -- 172.21.15.105:0/1005881318 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598035020 con 0x7f95a006b390 2024-03-08T08:28:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a4e09640 1 -- 172.21.15.105:0/1005881318 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598031640 con 0x7f95a006b390 2024-03-08T08:28:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a4e09640 1 -- 172.21.15.105:0/1005881318 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959802d040 con 0x7f95a006b390 2024-03-08T08:28:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a806f640 1 -- 172.21.15.105:0/1005881318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a006b390 msgr2=0x7f95a00ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a806f640 1 --2- 172.21.15.105:0/1005881318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a006b390 0x7f95a00ad970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9598009870 tx=0x7f959802b8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a806f640 1 -- 172.21.15.105:0/1005881318 shutdown_connections 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a806f640 1 --2- 172.21.15.105:0/1005881318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a006b390 0x7f95a00ad970 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a806f640 1 --2- 172.21.15.105:0/1005881318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a015df20 0x7f95a006c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a806f640 1 -- 172.21.15.105:0/1005881318 >> 172.21.15.105:0/1005881318 conn(0x7f95a006bd90 msgr2=0x7f95a006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a806f640 1 -- 172.21.15.105:0/1005881318 shutdown_connections 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.065+0000 7f95a806f640 1 -- 172.21.15.105:0/1005881318 wait complete. 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.066+0000 7f95a806f640 1 Processor -- start 2024-03-08T08:28:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.066+0000 7f95a806f640 1 -- start start 2024-03-08T08:28:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a806f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a006b390 0x7f95a01456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a806f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a00b5990 0x7f95a0154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a806f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a015df20 0x7f95a00ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a806f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00adeb0 con 0x7f95a00b5990 2024-03-08T08:28:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a806f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a006e0e0 con 0x7f95a015df20 2024-03-08T08:28:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a806f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00ae030 con 0x7f95a006b390 2024-03-08T08:28:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a560a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a00b5990 0x7f95a0154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a560a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a00b5990 0x7f95a0154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59066/0 (socket says 172.21.15.105:59066) 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a560a640 1 -- 172.21.15.105:0/1190748517 learned_addr learned my addr 172.21.15.105:0/1190748517 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a5e0b640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a006b390 0x7f95a01456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.067+0000 7f95a660c640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a015df20 0x7f95a00ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95a560a640 1 -- 172.21.15.105:0/1190748517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a006b390 msgr2=0x7f95a01456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95a560a640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a006b390 0x7f95a01456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95a560a640 1 -- 172.21.15.105:0/1190748517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a015df20 msgr2=0x7f95a00ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95a560a640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a015df20 0x7f95a00ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95a560a640 1 -- 172.21.15.105:0/1190748517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a0145fd0 con 0x7f95a00b5990 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95a560a640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a00b5990 0x7f95a0154060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f959000c450 tx=0x7f959000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95967fc640 1 -- 172.21.15.105:0/1190748517 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95900052d0 con 0x7f95a00b5990 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95967fc640 1 -- 172.21.15.105:0/1190748517 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590005470 con 0x7f95a00b5990 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a01462b0 con 0x7f95a00b5990 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95967fc640 1 -- 172.21.15.105:0/1190748517 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590012870 con 0x7f95a00b5990 2024-03-08T08:28:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.068+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a0146870 con 0x7f95a00b5990 2024-03-08T08:28:04.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.069+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a0075120 con 0x7f95a00b5990 2024-03-08T08:28:04.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.069+0000 7f95967fc640 1 -- 172.21.15.105:0/1190748517 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9590012a10 con 0x7f95a00b5990 2024-03-08T08:28:04.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.070+0000 7f95967fc640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f958406b4d0 0x7f958406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.070+0000 7f95967fc640 1 -- 172.21.15.105:0/1190748517 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9590089cb0 con 0x7f95a00b5990 2024-03-08T08:28:04.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.070+0000 7f95a5e0b640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f958406b4d0 0x7f958406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.070+0000 7f95a5e0b640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f958406b4d0 0x7f958406d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f959c009fd0 tx=0x7f959c0097f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.074+0000 7f95967fc640 1 -- 172.21.15.105:0/1190748517 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95900584e0 con 0x7f95a00b5990 2024-03-08T08:28:04.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.243+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95a00d0100 con 0x7f95a00b5990 2024-03-08T08:28:04.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.244+0000 7f95967fc640 1 -- 172.21.15.105:0/1190748517 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f959005c580 con 0x7f95a00b5990 2024-03-08T08:28:04.245 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:04.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f958406b4d0 msgr2=0x7f958406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f958406b4d0 0x7f958406d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f959c009fd0 tx=0x7f959c0097f0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a00b5990 msgr2=0x7f95a0154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a00b5990 0x7f95a0154060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f959000c450 tx=0x7f959000c910 comp rx=0 tx=0).stop 2024-03-08T08:28:04.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 shutdown_connections 2024-03-08T08:28:04.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f958406b4d0 0x7f958406d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a015df20 0x7f95a00ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95a00b5990 0x7f95a0154060 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 --2- 172.21.15.105:0/1190748517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a006b390 0x7f95a01456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 >> 172.21.15.105:0/1190748517 conn(0x7f95a006bd90 msgr2=0x7f95a00b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:04.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 shutdown_connections 2024-03-08T08:28:04.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.246+0000 7f95a806f640 1 -- 172.21.15.105:0/1190748517 wait complete. 2024-03-08T08:28:04.257 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:04.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.359+0000 7f69390f4640 1 Processor -- start 2024-03-08T08:28:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f69390f4640 1 -- start start 2024-03-08T08:28:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f69390f4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69340d2530 0x7f69340b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f69390f4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f693415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f69390f4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69340d48f0 con 0x7f69340d43e0 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f69390f4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340d4bb0 con 0x7f69340d2530 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f69390f4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69340ceb30 con 0x7f69340d2910 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f6932d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69340d2530 0x7f69340b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f6932d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f69340d2530 0x7f69340b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59068/0 (socket says 172.21.15.105:59068) 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.360+0000 7f6932d9d640 1 -- 172.21.15.105:0/2729627732 learned_addr learned my addr 172.21.15.105:0/2729627732 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.361+0000 7f693359e640 1 --2- 172.21.15.105:0/2729627732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f693415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.361+0000 7f6932d9d640 1 -- 172.21.15.105:0/2729627732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 msgr2=0x7f693415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.361+0000 7f6932d9d640 1 --2- 172.21.15.105:0/2729627732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f693415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.361+0000 7f6932d9d640 1 -- 172.21.15.105:0/2729627732 >> v1:172.21.15.144:6789/0 conn(0x7f69340d43e0 legacy=0x7f69340b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.361+0000 7f6932d9d640 1 -- 172.21.15.105:0/2729627732 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340cee10 con 0x7f69340d2530 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.361+0000 7f693359e640 1 --2- 172.21.15.105:0/2729627732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f693415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.361+0000 7f6932d9d640 1 --2- 172.21.15.105:0/2729627732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f69340b5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6928009870 tx=0x7f692802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b744f9f545df002 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f6931d9b640 1 -- 172.21.15.105:0/2729627732 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928035020 con 0x7f69340d2530 2024-03-08T08:28:04.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f6931d9b640 1 -- 172.21.15.105:0/2729627732 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6928031640 con 0x7f69340d2530 2024-03-08T08:28:04.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f6931d9b640 1 -- 172.21.15.105:0/2729627732 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692802d040 con 0x7f69340d2530 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f69390f4640 1 -- 172.21.15.105:0/2729627732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 msgr2=0x7f69340b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f69390f4640 1 --2- 172.21.15.105:0/2729627732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f69340b5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6928009870 tx=0x7f692802b8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f69390f4640 1 -- 172.21.15.105:0/2729627732 shutdown_connections 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f69390f4640 1 --2- 172.21.15.105:0/2729627732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f693415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f69390f4640 1 --2- 172.21.15.105:0/2729627732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f69340b5170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f69390f4640 1 -- 172.21.15.105:0/2729627732 >> 172.21.15.105:0/2729627732 conn(0x7f693409cfb0 msgr2=0x7f69340d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f69390f4640 1 -- 172.21.15.105:0/2729627732 shutdown_connections 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.362+0000 7f69390f4640 1 -- 172.21.15.105:0/2729627732 wait complete. 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f69390f4640 1 Processor -- start 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f69390f4640 1 -- start start 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f69390f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f6934158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f69390f4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f69340cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f69390f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69340d43e0 0x7f69340cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f69390f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693415e340 con 0x7f69340d2530 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f69390f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69340b3080 con 0x7f69340d43e0 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f69390f4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693415e4c0 con 0x7f69340d2910 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f6932d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f6934158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f6932d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f6934158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59082/0 (socket says 172.21.15.105:59082) 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.364+0000 7f6932d9d640 1 -- 172.21.15.105:0/3433180531 learned_addr learned my addr 172.21.15.105:0/3433180531 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f693259c640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f69340cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f693359e640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69340d43e0 0x7f69340cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f6932d9d640 1 -- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 msgr2=0x7f69340cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f6932d9d640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f69340cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f6932d9d640 1 -- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69340d43e0 msgr2=0x7f69340cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f6932d9d640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69340d43e0 0x7f69340cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f6932d9d640 1 -- 172.21.15.105:0/3433180531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340b77c0 con 0x7f69340d2530 2024-03-08T08:28:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f693259c640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f69340cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f693359e640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69340d43e0 0x7f69340cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f6932d9d640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f6934158240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6928000c00 tx=0x7f69280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f691b7fe640 1 -- 172.21.15.105:0/3433180531 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928040070 con 0x7f69340d2530 2024-03-08T08:28:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f691b7fe640 1 -- 172.21.15.105:0/3433180531 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69280399f0 con 0x7f69340d2530 2024-03-08T08:28:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69340b7aa0 con 0x7f69340d2530 2024-03-08T08:28:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f691b7fe640 1 -- 172.21.15.105:0/3433180531 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928035040 con 0x7f69340d2530 2024-03-08T08:28:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.365+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69340b8060 con 0x7f69340d2530 2024-03-08T08:28:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.366+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6934075120 con 0x7f69340d2530 2024-03-08T08:28:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.366+0000 7f691b7fe640 1 -- 172.21.15.105:0/3433180531 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f692804b060 con 0x7f69340d2530 2024-03-08T08:28:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.367+0000 7f691b7fe640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f692006b500 0x7f692006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.367+0000 7f691b7fe640 1 -- 172.21.15.105:0/3433180531 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69280abb60 con 0x7f69340d2530 2024-03-08T08:28:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.367+0000 7f693259c640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f692006b500 0x7f692006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.367+0000 7f693259c640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f692006b500 0x7f692006d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69340b4c70 tx=0x7f6924009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.371+0000 7f691b7fe640 1 -- 172.21.15.105:0/3433180531 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f692807a9e0 con 0x7f69340d2530 2024-03-08T08:28:04.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.539+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f693409d500 con 0x7f69340d2530 2024-03-08T08:28:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.540+0000 7f691b7fe640 1 -- 172.21.15.105:0/3433180531 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f692807a2e0 con 0x7f69340d2530 2024-03-08T08:28:04.541 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f692006b500 msgr2=0x7f692006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f692006b500 0x7f692006d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69340b4c70 tx=0x7f6924009040 comp rx=0 tx=0).stop 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 msgr2=0x7f6934158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f6934158240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6928000c00 tx=0x7f69280046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 shutdown_connections 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f692006b500 0x7f692006d9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69340d43e0 0x7f69340cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69340d2910 0x7f69340cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 --2- 172.21.15.105:0/3433180531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69340d2530 0x7f6934158240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 >> 172.21.15.105:0/3433180531 conn(0x7f693409cfb0 msgr2=0x7f693406b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 shutdown_connections 2024-03-08T08:28:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.542+0000 7f69390f4640 1 -- 172.21.15.105:0/3433180531 wait complete. 2024-03-08T08:28:04.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.655+0000 7f8c190ec640 1 Processor -- start 2024-03-08T08:28:04.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.655+0000 7f8c190ec640 1 -- start start 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.655+0000 7f8c190ec640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c141615d0 0x7f8c140b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.655+0000 7f8c190ec640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c140d05a0 0x7f8c140d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.655+0000 7f8c190ec640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8c140ce8e0 0x7f8c140ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.655+0000 7f8c190ec640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c140d0f40 con 0x7f8c141615d0 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.655+0000 7f8c190ec640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c140cf660 con 0x7f8c140ce8e0 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.655+0000 7f8c190ec640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140cf940 con 0x7f8c140d05a0 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c12d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c141615d0 0x7f8c140b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c12d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c141615d0 0x7f8c140b6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59090/0 (socket says 172.21.15.105:59090) 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c12d9d640 1 -- 172.21.15.105:0/1907953711 learned_addr learned my addr 172.21.15.105:0/1907953711 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c1259c640 1 --2- 172.21.15.105:0/1907953711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c140d05a0 0x7f8c140d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c1359e640 1 --2- 172.21.15.105:0/1907953711 >> v2:172.21.15.144:3300/0 conn(0x7f8c140ce8e0 0x7f8c140ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c12d9d640 1 -- 172.21.15.105:0/1907953711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c140d05a0 msgr2=0x7f8c140d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c12d9d640 1 --2- 172.21.15.105:0/1907953711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c140d05a0 0x7f8c140d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c12d9d640 1 -- 172.21.15.105:0/1907953711 >> v2:172.21.15.144:3300/0 conn(0x7f8c140ce8e0 msgr2=0x7f8c140ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c12d9d640 1 --2- 172.21.15.105:0/1907953711 >> v2:172.21.15.144:3300/0 conn(0x7f8c140ce8e0 0x7f8c140ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c12d9d640 1 -- 172.21.15.105:0/1907953711 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c140cfc20 con 0x7f8c141615d0 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c1359e640 1 --2- 172.21.15.105:0/1907953711 >> v2:172.21.15.144:3300/0 conn(0x7f8c140ce8e0 0x7f8c140ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.656+0000 7f8c1259c640 1 --2- 172.21.15.105:0/1907953711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c140d05a0 0x7f8c140d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c12d9d640 1 --2- 172.21.15.105:0/1907953711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c141615d0 0x7f8c140b6b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c140c7470 tx=0x7f8c0402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180d5bbb49cdd106 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c11d9b640 1 -- 172.21.15.105:0/1907953711 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0402ca70 con 0x7f8c141615d0 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c11d9b640 1 -- 172.21.15.105:0/1907953711 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0402cc10 con 0x7f8c141615d0 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c11d9b640 1 -- 172.21.15.105:0/1907953711 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c040343f0 con 0x7f8c141615d0 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 -- 172.21.15.105:0/1907953711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c141615d0 msgr2=0x7f8c140b6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 --2- 172.21.15.105:0/1907953711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c141615d0 0x7f8c140b6b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c140c7470 tx=0x7f8c0402bed0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 -- 172.21.15.105:0/1907953711 shutdown_connections 2024-03-08T08:28:04.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 --2- 172.21.15.105:0/1907953711 >> v2:172.21.15.144:3300/0 conn(0x7f8c140ce8e0 0x7f8c140ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 --2- 172.21.15.105:0/1907953711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c140d05a0 0x7f8c140d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 --2- 172.21.15.105:0/1907953711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c141615d0 0x7f8c140b6b20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 -- 172.21.15.105:0/1907953711 >> 172.21.15.105:0/1907953711 conn(0x7f8c1409cfa0 msgr2=0x7f8c140d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 -- 172.21.15.105:0/1907953711 shutdown_connections 2024-03-08T08:28:04.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.657+0000 7f8c190ec640 1 -- 172.21.15.105:0/1907953711 wait complete. 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.659+0000 7f8c190ec640 1 Processor -- start 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.659+0000 7f8c190ec640 1 -- start start 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c190ec640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c140ce8e0 0x7f8c140cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c190ec640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c140d05a0 0x7f8c140c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c190ec640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c141615d0 0x7f8c140c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c190ec640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140b7060 con 0x7f8c140d05a0 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c190ec640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140d1c20 con 0x7f8c140ce8e0 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c190ec640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141619b0 con 0x7f8c141615d0 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c140d05a0 0x7f8c140c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c140d05a0 0x7f8c140c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59102/0 (socket says 172.21.15.105:59102) 2024-03-08T08:28:04.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 -- 172.21.15.105:0/47574651 learned_addr learned my addr 172.21.15.105:0/47574651 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1359e640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c141615d0 0x7f8c140c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c12d9d640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c140ce8e0 0x7f8c140cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 -- 172.21.15.105:0/47574651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c141615d0 msgr2=0x7f8c140c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c141615d0 0x7f8c140c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 -- 172.21.15.105:0/47574651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c140ce8e0 msgr2=0x7f8c140cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c140ce8e0 0x7f8c140cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 -- 172.21.15.105:0/47574651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c140cce90 con 0x7f8c140d05a0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c1259c640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c140d05a0 0x7f8c140c7b60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800e7e0 tx=0x7f8c0800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8bfb7fe640 1 -- 172.21.15.105:0/47574651 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800f040 con 0x7f8c140d05a0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8bfb7fe640 1 -- 172.21.15.105:0/47574651 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08004030 con 0x7f8c140d05a0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c140ccbb0 con 0x7f8c140d05a0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8bfb7fe640 1 -- 172.21.15.105:0/47574651 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08004400 con 0x7f8c140d05a0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.660+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c14159f30 con 0x7f8c140d05a0 2024-03-08T08:28:04.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.661+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c14075120 con 0x7f8c140d05a0 2024-03-08T08:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.662+0000 7f8bfb7fe640 1 -- 172.21.15.105:0/47574651 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c0800c6d0 con 0x7f8c140d05a0 2024-03-08T08:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.662+0000 7f8bfb7fe640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8c0006b4d0 0x7f8c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.662+0000 7f8bfb7fe640 1 -- 172.21.15.105:0/47574651 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c08089fc0 con 0x7f8c140d05a0 2024-03-08T08:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.662+0000 7f8c12d9d640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8c0006b4d0 0x7f8c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.662+0000 7f8c12d9d640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8c0006b4d0 0x7f8c0006d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c04035000 tx=0x7f8c04030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.666+0000 7f8bfb7fe640 1 -- 172.21.15.105:0/47574651 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c08017020 con 0x7f8c140d05a0 2024-03-08T08:28:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.836+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c140d1350 con 0x7f8c140d05a0 2024-03-08T08:28:04.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.836+0000 7f8bfb7fe640 1 -- 172.21.15.105:0/47574651 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c08058e40 con 0x7f8c140d05a0 2024-03-08T08:28:04.837 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:04.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8c0006b4d0 msgr2=0x7f8c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8c0006b4d0 0x7f8c0006d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c04035000 tx=0x7f8c04030000 comp rx=0 tx=0).stop 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c140d05a0 msgr2=0x7f8c140c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c140d05a0 0x7f8c140c7b60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800e7e0 tx=0x7f8c0800eca0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 shutdown_connections 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8c0006b4d0 0x7f8c0006d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c141615d0 0x7f8c140c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c140d05a0 0x7f8c140c7b60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 --2- 172.21.15.105:0/47574651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c140ce8e0 0x7f8c140cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 >> 172.21.15.105:0/47574651 conn(0x7f8c1409cfa0 msgr2=0x7f8c1415c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 shutdown_connections 2024-03-08T08:28:04.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.838+0000 7f8c190ec640 1 -- 172.21.15.105:0/47574651 wait complete. 2024-03-08T08:28:04.850 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.949+0000 7f371bb48640 1 Processor -- start 2024-03-08T08:28:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.949+0000 7f371bb48640 1 -- start start 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f371bb48640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140ce330 0x7f37140ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f371bb48640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37140cfb40 0x7f37140cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f371bb48640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37140b29e0 con 0x7f3714161520 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f371bb48640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37140d1930 con 0x7f37140cfb40 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f371bb48640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37140cecd0 con 0x7f37140ce330 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f371a0e5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37140cfb40 0x7f37140cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f37198e4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3714161520 0x7f37140b2d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41868/0 (socket says 172.21.15.105:41868) 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f371a0e5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37140cfb40 0x7f37140cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59118/0 (socket says 172.21.15.105:59118) 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f37198e4640 1 -- 172.21.15.105:0/4166219349 learned_addr learned my addr 172.21.15.105:0/4166219349 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.950+0000 7f37190e3640 1 --2- 172.21.15.105:0/4166219349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140ce330 0x7f37140ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f371a0e5640 1 -- 172.21.15.105:0/4166219349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140ce330 msgr2=0x7f37140ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:04.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f371a0e5640 1 --2- 172.21.15.105:0/4166219349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140ce330 0x7f37140ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f371a0e5640 1 -- 172.21.15.105:0/4166219349 >> v1:172.21.15.105:6789/0 conn(0x7f3714161520 legacy=0x7f37140b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f371a0e5640 1 -- 172.21.15.105:0/4166219349 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140d1650 con 0x7f37140cfb40 2024-03-08T08:28:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f37188e2640 1 -- 172.21.15.105:0/4166219349 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525424057 0 0) 0x7f37140b29e0 con 0x7f3714161520 2024-03-08T08:28:04.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f37190e3640 1 --2- 172.21.15.105:0/4166219349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140ce330 0x7f37140ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f371a0e5640 1 --2- 172.21.15.105:0/4166219349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140cfb40 0x7f37140cffd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f370c009d90 tx=0x7f370c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db5742a70e5b3e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f37188e2640 1 -- 172.21.15.105:0/4166219349 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c039070 con 0x7f37140cfb40 2024-03-08T08:28:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f37188e2640 1 -- 172.21.15.105:0/4166219349 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c02cc50 con 0x7f37140cfb40 2024-03-08T08:28:04.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.951+0000 7f37188e2640 1 -- 172.21.15.105:0/4166219349 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c034440 con 0x7f37140cfb40 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.952+0000 7f371bb48640 1 -- 172.21.15.105:0/4166219349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140cfb40 msgr2=0x7f37140cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.952+0000 7f371bb48640 1 --2- 172.21.15.105:0/4166219349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140cfb40 0x7f37140cffd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f370c009d90 tx=0x7f370c02bea0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.952+0000 7f371bb48640 1 -- 172.21.15.105:0/4166219349 shutdown_connections 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.952+0000 7f371bb48640 1 --2- 172.21.15.105:0/4166219349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140cfb40 0x7f37140cffd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.952+0000 7f371bb48640 1 --2- 172.21.15.105:0/4166219349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140ce330 0x7f37140ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.952+0000 7f371bb48640 1 -- 172.21.15.105:0/4166219349 >> 172.21.15.105:0/4166219349 conn(0x7f37140b36a0 msgr2=0x7f37140b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.952+0000 7f371bb48640 1 -- 172.21.15.105:0/4166219349 shutdown_connections 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.952+0000 7f371bb48640 1 -- 172.21.15.105:0/4166219349 wait complete. 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.953+0000 7f371bb48640 1 Processor -- start 2024-03-08T08:28:04.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.953+0000 7f371bb48640 1 -- start start 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f371bb48640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140ce330 0x7f3714154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f371bb48640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140cfb40 0x7f371415fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f371bb48640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3714161520 0x7f37141600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f371bb48640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371406ca40 con 0x7f37140ce330 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f371bb48640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371406c770 con 0x7f3714161520 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f371bb48640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371406ced0 con 0x7f37140cfb40 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140ce330 0x7f3714154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140ce330 0x7f3714154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59120/0 (socket says 172.21.15.105:59120) 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 -- 172.21.15.105:0/434321594 learned_addr learned my addr 172.21.15.105:0/434321594 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37190e3640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140cfb40 0x7f371415fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 -- 172.21.15.105:0/434321594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140cfb40 msgr2=0x7f371415fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140cfb40 0x7f371415fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 -- 172.21.15.105:0/434321594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3714161520 msgr2=0x7f37141600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3714161520 0x7f37141600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 -- 172.21.15.105:0/434321594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140c9bb0 con 0x7f37140ce330 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.954+0000 7f37198e4640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140ce330 0x7f3714154ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f371000c430 tx=0x7f371000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.955+0000 7f370a7fc640 1 -- 172.21.15.105:0/434321594 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37100074e0 con 0x7f37140ce330 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.955+0000 7f370a7fc640 1 -- 172.21.15.105:0/434321594 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37100047e0 con 0x7f37140ce330 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.955+0000 7f370a7fc640 1 -- 172.21.15.105:0/434321594 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710004bd0 con 0x7f37140ce330 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.955+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37140c9e90 con 0x7f37140ce330 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.955+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37140ca730 con 0x7f37140ce330 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.956+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37140ca060 con 0x7f37140ce330 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.956+0000 7f370a7fc640 1 -- 172.21.15.105:0/434321594 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3710011070 con 0x7f37140ce330 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.956+0000 7f370a7fc640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f36f806b4d0 0x7f36f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.957+0000 7f37190e3640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f36f806b4d0 0x7f36f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:04.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.957+0000 7f370a7fc640 1 -- 172.21.15.105:0/434321594 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3710089e30 con 0x7f37140ce330 2024-03-08T08:28:04.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.957+0000 7f37190e3640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f36f806b4d0 0x7f36f806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3704006fd0 tx=0x7f3704008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:04.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:04.961+0000 7f370a7fc640 1 -- 172.21.15.105:0/434321594 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3710058d40 con 0x7f37140ce330 2024-03-08T08:28:05.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.133+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3714075120 con 0x7f37140ce330 2024-03-08T08:28:05.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.135+0000 7f370a7fc640 1 -- 172.21.15.105:0/434321594 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f3710058640 con 0x7f37140ce330 2024-03-08T08:28:05.136 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:05.136 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f36f806b4d0 msgr2=0x7f36f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f36f806b4d0 0x7f36f806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3704006fd0 tx=0x7f3704008040 comp rx=0 tx=0).stop 2024-03-08T08:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140ce330 msgr2=0x7f3714154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140ce330 0x7f3714154ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f371000c430 tx=0x7f371000c8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 shutdown_connections 2024-03-08T08:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f36f806b4d0 0x7f36f806d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3714161520 0x7f37141600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37140cfb40 0x7f371415fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 --2- 172.21.15.105:0/434321594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37140ce330 0x7f3714154ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 >> 172.21.15.105:0/434321594 conn(0x7f37140b36a0 msgr2=0x7f37140d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 shutdown_connections 2024-03-08T08:28:05.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.137+0000 7f371bb48640 1 -- 172.21.15.105:0/434321594 wait complete. 2024-03-08T08:28:05.149 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:05.149 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:05.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.251+0000 7f9abe2d4640 1 Processor -- start 2024-03-08T08:28:05.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.251+0000 7f9abe2d4640 1 -- start start 2024-03-08T08:28:05.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9abe2d4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ab8161520 0x7f9ab80b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9abe2d4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80ce330 0x7f9ab80ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9abe2d4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab80b29e0 con 0x7f9ab80b14f0 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9abe2d4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab80d1930 con 0x7f9ab8161520 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9abe2d4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80cecd0 con 0x7f9ab80ce330 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9ab7fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ab8161520 0x7f9ab80b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9ab77fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80ce330 0x7f9ab80ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9ab7fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ab8161520 0x7f9ab80b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:50556/0 (socket says 172.21.15.105:50556) 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9ab7fff640 1 -- 172.21.15.105:0/2512419881 learned_addr learned my addr 172.21.15.105:0/2512419881 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9abc871640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9ab80b14f0 0x7f9ab80b1980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:57576/0 (socket says 172.21.15.105:57576) 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9ab7fff640 1 -- 172.21.15.105:0/2512419881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80ce330 msgr2=0x7f9ab80ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9ab7fff640 1 --2- 172.21.15.105:0/2512419881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80ce330 0x7f9ab80ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9ab7fff640 1 -- 172.21.15.105:0/2512419881 >> v1:172.21.15.144:6789/0 conn(0x7f9ab80b14f0 legacy=0x7f9ab80b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.252+0000 7f9ab7fff640 1 -- 172.21.15.105:0/2512419881 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab80d1650 con 0x7f9ab8161520 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.253+0000 7f9ab7fff640 1 --2- 172.21.15.105:0/2512419881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8161520 0x7f9ab80b2d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009870 tx=0x7f9aa802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35395a0dbba9ebf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.253+0000 7f9ab6ffd640 1 -- 172.21.15.105:0/2512419881 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8034070 con 0x7f9ab8161520 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.253+0000 7f9ab6ffd640 1 -- 172.21.15.105:0/2512419881 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa8030840 con 0x7f9ab8161520 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.253+0000 7f9ab6ffd640 1 -- 172.21.15.105:0/2512419881 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8030c10 con 0x7f9ab8161520 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.254+0000 7f9abe2d4640 1 -- 172.21.15.105:0/2512419881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8161520 msgr2=0x7f9ab80b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.254+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/2512419881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8161520 0x7f9ab80b2d90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009870 tx=0x7f9aa802b980 comp rx=0 tx=0).stop 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.254+0000 7f9abe2d4640 1 -- 172.21.15.105:0/2512419881 shutdown_connections 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.254+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/2512419881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80ce330 0x7f9ab80ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.254+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/2512419881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab8161520 0x7f9ab80b2d90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.254+0000 7f9abe2d4640 1 -- 172.21.15.105:0/2512419881 >> 172.21.15.105:0/2512419881 conn(0x7f9ab80b36a0 msgr2=0x7f9ab80b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:05.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.254+0000 7f9abe2d4640 1 -- 172.21.15.105:0/2512419881 shutdown_connections 2024-03-08T08:28:05.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.254+0000 7f9abe2d4640 1 -- 172.21.15.105:0/2512419881 wait complete. 2024-03-08T08:28:05.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.255+0000 7f9abe2d4640 1 Processor -- start 2024-03-08T08:28:05.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.255+0000 7f9abe2d4640 1 -- start start 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abe2d4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80b14f0 0x7f9ab81606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abe2d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab80ce330 0x7f9ab80c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abe2d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ab8161520 0x7f9ab81e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abe2d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab806ca40 con 0x7f9ab8161520 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abe2d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab806c770 con 0x7f9ab80ce330 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abe2d4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b32d0 con 0x7f9ab80b14f0 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abc871640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ab8161520 0x7f9ab81e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abc871640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ab8161520 0x7f9ab81e9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59136/0 (socket says 172.21.15.105:59136) 2024-03-08T08:28:05.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abc871640 1 -- 172.21.15.105:0/3018065090 learned_addr learned my addr 172.21.15.105:0/3018065090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9ab77fe640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab80ce330 0x7f9ab80c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9ab7fff640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80b14f0 0x7f9ab81606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abc871640 1 -- 172.21.15.105:0/3018065090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80b14f0 msgr2=0x7f9ab81606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abc871640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80b14f0 0x7f9ab81606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abc871640 1 -- 172.21.15.105:0/3018065090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab80ce330 msgr2=0x7f9ab80c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abc871640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab80ce330 0x7f9ab80c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.256+0000 7f9abc871640 1 -- 172.21.15.105:0/3018065090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab80c6fd0 con 0x7f9ab8161520 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.257+0000 7f9abc871640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ab8161520 0x7f9ab81e9a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000c780 tx=0x7f9ab000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.257+0000 7f9ab4ff9640 1 -- 172.21.15.105:0/3018065090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0017070 con 0x7f9ab8161520 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.257+0000 7f9ab4ff9640 1 -- 172.21.15.105:0/3018065090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab000a640 con 0x7f9ab8161520 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.257+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab80c72b0 con 0x7f9ab8161520 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.257+0000 7f9ab4ff9640 1 -- 172.21.15.105:0/3018065090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000aa30 con 0x7f9ab8161520 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.257+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab80c7870 con 0x7f9ab8161520 2024-03-08T08:28:05.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.258+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab8075120 con 0x7f9ab8161520 2024-03-08T08:28:05.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.258+0000 7f9ab4ff9640 1 -- 172.21.15.105:0/3018065090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ab001a070 con 0x7f9ab8161520 2024-03-08T08:28:05.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.259+0000 7f9ab4ff9640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9aa406b4d0 0x7f9aa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.259+0000 7f9ab4ff9640 1 -- 172.21.15.105:0/3018065090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ab00896f0 con 0x7f9ab8161520 2024-03-08T08:28:05.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.259+0000 7f9ab7fff640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9aa406b4d0 0x7f9aa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.259+0000 7f9ab7fff640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9aa406b4d0 0x7f9aa406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8036000 tx=0x7f9aa803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.263+0000 7f9ab4ff9640 1 -- 172.21.15.105:0/3018065090 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ab0057f30 con 0x7f9ab8161520 2024-03-08T08:28:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.432+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ab80b79d0 con 0x7f9ab8161520 2024-03-08T08:28:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.432+0000 7f9ab4ff9640 1 -- 172.21.15.105:0/3018065090 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ab005bfd0 con 0x7f9ab8161520 2024-03-08T08:28:05.433 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9aa406b4d0 msgr2=0x7f9aa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9aa406b4d0 0x7f9aa406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8036000 tx=0x7f9aa803b040 comp rx=0 tx=0).stop 2024-03-08T08:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ab8161520 msgr2=0x7f9ab81e9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ab8161520 0x7f9ab81e9a20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000c780 tx=0x7f9ab000cc40 comp rx=0 tx=0).stop 2024-03-08T08:28:05.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 shutdown_connections 2024-03-08T08:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9aa406b4d0 0x7f9aa406d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ab8161520 0x7f9ab81e9a20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab80ce330 0x7f9ab80c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 --2- 172.21.15.105:0/3018065090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80b14f0 0x7f9ab81606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 >> 172.21.15.105:0/3018065090 conn(0x7f9ab80b36a0 msgr2=0x7f9ab806fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 shutdown_connections 2024-03-08T08:28:05.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.434+0000 7f9abe2d4640 1 -- 172.21.15.105:0/3018065090 wait complete. 2024-03-08T08:28:05.446 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:05.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.548+0000 7fde6506b640 1 Processor -- start 2024-03-08T08:28:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde6506b640 1 -- start start 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde6506b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde60072440 0x7fde60071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde6506b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 0x7fde6015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde6506b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6015b940 con 0x7fde6015b4a0 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde6506b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6015bc20 con 0x7fde60072440 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde6506b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde60070840 con 0x7fde600b4d00 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde5ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde60072440 0x7fde60071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde5ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde60072440 0x7fde60071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59152/0 (socket says 172.21.15.105:59152) 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde5ed9d640 1 -- 172.21.15.105:0/1936607354 learned_addr learned my addr 172.21.15.105:0/1936607354 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.549+0000 7fde5f59e640 1 --1- 172.21.15.105:0/1936607354 >> v1:172.21.15.105:6789/0 conn(0x7fde6015b4a0 0x7fde6006e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41874/0 (socket says 172.21.15.105:41874) 2024-03-08T08:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.550+0000 7fde57fff640 1 --2- 172.21.15.105:0/1936607354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 0x7fde6015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.550+0000 7fde5ed9d640 1 -- 172.21.15.105:0/1936607354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 msgr2=0x7fde6015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.550+0000 7fde5ed9d640 1 --2- 172.21.15.105:0/1936607354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 0x7fde6015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.550+0000 7fde5ed9d640 1 -- 172.21.15.105:0/1936607354 >> v1:172.21.15.105:6789/0 conn(0x7fde6015b4a0 legacy=0x7fde6006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.550+0000 7fde5ed9d640 1 -- 172.21.15.105:0/1936607354 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60070b20 con 0x7fde60072440 2024-03-08T08:28:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.550+0000 7fde5e59c640 1 -- 172.21.15.105:0/1936607354 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512485197 0 0) 0x7fde6015b940 con 0x7fde6015b4a0 2024-03-08T08:28:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.550+0000 7fde5ed9d640 1 --2- 172.21.15.105:0/1936607354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde60071020 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde50009870 tx=0x7fde5002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43957eed0952c15 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde5e59c640 1 -- 172.21.15.105:0/1936607354 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50034070 con 0x7fde60072440 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde5e59c640 1 -- 172.21.15.105:0/1936607354 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde50030840 con 0x7fde60072440 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde5e59c640 1 -- 172.21.15.105:0/1936607354 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50030c10 con 0x7fde60072440 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde6506b640 1 -- 172.21.15.105:0/1936607354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 msgr2=0x7fde60071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde6506b640 1 --2- 172.21.15.105:0/1936607354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde60071020 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde50009870 tx=0x7fde5002b980 comp rx=0 tx=0).stop 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde6506b640 1 -- 172.21.15.105:0/1936607354 shutdown_connections 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde6506b640 1 --2- 172.21.15.105:0/1936607354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 0x7fde6015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde6506b640 1 --2- 172.21.15.105:0/1936607354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde60071020 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde6506b640 1 -- 172.21.15.105:0/1936607354 >> 172.21.15.105:0/1936607354 conn(0x7fde600b1740 msgr2=0x7fde600b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde6506b640 1 -- 172.21.15.105:0/1936607354 shutdown_connections 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.551+0000 7fde6506b640 1 -- 172.21.15.105:0/1936607354 wait complete. 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde6506b640 1 Processor -- start 2024-03-08T08:28:05.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde6506b640 1 -- start start 2024-03-08T08:28:05.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde6506b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde601eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde6506b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 0x7fde6013b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde6506b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde6015b4a0 0x7fde6013be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde6506b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde600729a0 con 0x7fde60072440 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde6506b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde60072820 con 0x7fde6015b4a0 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde6506b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6006e7b0 con 0x7fde600b4d00 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde5ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde601eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde5ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde601eff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59158/0 (socket says 172.21.15.105:59158) 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde5ed9d640 1 -- 172.21.15.105:0/896842441 learned_addr learned my addr 172.21.15.105:0/896842441 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde57fff640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 0x7fde6013b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.553+0000 7fde5f59e640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde6015b4a0 0x7fde6013be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde5ed9d640 1 -- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 msgr2=0x7fde6013b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde5ed9d640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 0x7fde6013b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde5ed9d640 1 -- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde6015b4a0 msgr2=0x7fde6013be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde5ed9d640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde6015b4a0 0x7fde6013be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde5ed9d640 1 -- 172.21.15.105:0/896842441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60157f30 con 0x7fde60072440 2024-03-08T08:28:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde5f59e640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde6015b4a0 0x7fde6013be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde5ed9d640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde601eff00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde50009870 tx=0x7fde500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde577fe640 1 -- 172.21.15.105:0/896842441 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50034040 con 0x7fde60072440 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde577fe640 1 -- 172.21.15.105:0/896842441 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde50030640 con 0x7fde60072440 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde601573b0 con 0x7fde60072440 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde577fe640 1 -- 172.21.15.105:0/896842441 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50004030 con 0x7fde60072440 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.554+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde60157970 con 0x7fde60072440 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.555+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde60075120 con 0x7fde60072440 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.555+0000 7fde577fe640 1 -- 172.21.15.105:0/896842441 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde5004d060 con 0x7fde60072440 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.556+0000 7fde577fe640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fde4006b4d0 0x7fde4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.556+0000 7fde577fe640 1 -- 172.21.15.105:0/896842441 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde500adf50 con 0x7fde60072440 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.556+0000 7fde57fff640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fde4006b4d0 0x7fde4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.556+0000 7fde57fff640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fde4006b4d0 0x7fde4006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fde6015cb20 tx=0x7fde48009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.560+0000 7fde577fe640 1 -- 172.21.15.105:0/896842441 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde5007cf80 con 0x7fde60072440 2024-03-08T08:28:05.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.731+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde6013caa0 con 0x7fde60072440 2024-03-08T08:28:05.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.731+0000 7fde577fe640 1 -- 172.21.15.105:0/896842441 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde5007c880 con 0x7fde60072440 2024-03-08T08:28:05.733 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fde4006b4d0 msgr2=0x7fde4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fde4006b4d0 0x7fde4006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fde6015cb20 tx=0x7fde48009190 comp rx=0 tx=0).stop 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 msgr2=0x7fde601eff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde601eff00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde50009870 tx=0x7fde500046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 shutdown_connections 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fde4006b4d0 0x7fde4006d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde6015b4a0 0x7fde6013be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde600b4d00 0x7fde6013b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 --2- 172.21.15.105:0/896842441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde60072440 0x7fde601eff00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 >> 172.21.15.105:0/896842441 conn(0x7fde600b1740 msgr2=0x7fde6006e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 shutdown_connections 2024-03-08T08:28:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.733+0000 7fde6506b640 1 -- 172.21.15.105:0/896842441 wait complete. 2024-03-08T08:28:05.745 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.848+0000 7f47fcbac640 1 Processor -- start 2024-03-08T08:28:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.848+0000 7f47fcbac640 1 -- start start 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47fcbac640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f47f80d2530 0x7f47f80b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47fcbac640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f47f80d43e0 0x7f47f80b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47fcbac640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f80b3880 con 0x7f47f80d2910 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47fcbac640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f80d48f0 con 0x7f47f80d2530 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47fcbac640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f80d4bb0 con 0x7f47f80d43e0 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47f659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f47f80d2530 0x7f47f80b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47f659c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f47f80d2530 0x7f47f80b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59168/0 (socket says 172.21.15.105:59168) 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47f659c640 1 -- 172.21.15.105:0/1664284602 learned_addr learned my addr 172.21.15.105:0/1664284602 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:05.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47effff640 1 --2- 172.21.15.105:0/1664284602 >> v2:172.21.15.144:3300/0 conn(0x7f47f80d43e0 0x7f47f80b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47f659c640 1 -- 172.21.15.105:0/1664284602 >> v2:172.21.15.144:3300/0 conn(0x7f47f80d43e0 msgr2=0x7f47f80b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47f659c640 1 --2- 172.21.15.105:0/1664284602 >> v2:172.21.15.144:3300/0 conn(0x7f47f80d43e0 0x7f47f80b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47f659c640 1 -- 172.21.15.105:0/1664284602 >> v1:172.21.15.144:6789/0 conn(0x7f47f80d2910 legacy=0x7f47f815de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.849+0000 7f47f659c640 1 -- 172.21.15.105:0/1664284602 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80ceb30 con 0x7f47f80d2530 2024-03-08T08:28:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.850+0000 7f47f659c640 1 --2- 172.21.15.105:0/1664284602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d2530 0x7f47f80b5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f47f808cf60 tx=0x7f47e802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f17256874da596 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.850+0000 7f47f5d9b640 1 -- 172.21.15.105:0/1664284602 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e802e070 con 0x7f47f80d2530 2024-03-08T08:28:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.850+0000 7f47f5d9b640 1 -- 172.21.15.105:0/1664284602 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e8037070 con 0x7f47f80d2530 2024-03-08T08:28:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.850+0000 7f47f5d9b640 1 -- 172.21.15.105:0/1664284602 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8031a00 con 0x7f47f80d2530 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.851+0000 7f47fcbac640 1 -- 172.21.15.105:0/1664284602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d2530 msgr2=0x7f47f80b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.851+0000 7f47fcbac640 1 --2- 172.21.15.105:0/1664284602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d2530 0x7f47f80b5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f47f808cf60 tx=0x7f47e802b880 comp rx=0 tx=0).stop 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.851+0000 7f47fcbac640 1 -- 172.21.15.105:0/1664284602 shutdown_connections 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.851+0000 7f47fcbac640 1 --2- 172.21.15.105:0/1664284602 >> v2:172.21.15.144:3300/0 conn(0x7f47f80d43e0 0x7f47f80b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.851+0000 7f47fcbac640 1 --2- 172.21.15.105:0/1664284602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d2530 0x7f47f80b5170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.851+0000 7f47fcbac640 1 -- 172.21.15.105:0/1664284602 >> 172.21.15.105:0/1664284602 conn(0x7f47f809cfb0 msgr2=0x7f47f80d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.851+0000 7f47fcbac640 1 -- 172.21.15.105:0/1664284602 shutdown_connections 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.851+0000 7f47fcbac640 1 -- 172.21.15.105:0/1664284602 wait complete. 2024-03-08T08:28:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.852+0000 7f47fcbac640 1 Processor -- start 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.852+0000 7f47fcbac640 1 -- start start 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47fcbac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47f80d2530 0x7f47f80b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47fcbac640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47f80d2910 0x7f47f80c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47fcbac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d43e0 0x7f47f80ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47fcbac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80b3080 con 0x7f47f80d43e0 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47fcbac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80cf0f0 con 0x7f47f80d2530 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47fcbac640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f815e510 con 0x7f47f80d2910 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d43e0 0x7f47f80ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d43e0 0x7f47f80ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59180/0 (socket says 172.21.15.105:59180) 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 -- 172.21.15.105:0/874524060 learned_addr learned my addr 172.21.15.105:0/874524060 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f659c640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47f80d2530 0x7f47f80b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47effff640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47f80d2910 0x7f47f80c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 -- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47f80d2910 msgr2=0x7f47f80c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47f80d2910 0x7f47f80c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 -- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47f80d2530 msgr2=0x7f47f80b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47f80d2530 0x7f47f80b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 -- 172.21.15.105:0/874524060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80b96e0 con 0x7f47f80d43e0 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47effff640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47f80d2910 0x7f47f80c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47f6d9d640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d43e0 0x7f47f80ca3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c780 tx=0x7f47f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47eeffd640 1 -- 172.21.15.105:0/874524060 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f0004b60 con 0x7f47f80d43e0 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47eeffd640 1 -- 172.21.15.105:0/874524060 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f0004d00 con 0x7f47f80d43e0 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f80c93d0 con 0x7f47f80d43e0 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47eeffd640 1 -- 172.21.15.105:0/874524060 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f000aa90 con 0x7f47f80d43e0 2024-03-08T08:28:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.853+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f80c9990 con 0x7f47f80d43e0 2024-03-08T08:28:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.855+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f808d9c0 con 0x7f47f80d43e0 2024-03-08T08:28:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.855+0000 7f47eeffd640 1 -- 172.21.15.105:0/874524060 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47f000ac30 con 0x7f47f80d43e0 2024-03-08T08:28:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.855+0000 7f47eeffd640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f47d806b4d0 0x7f47d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.855+0000 7f47eeffd640 1 -- 172.21.15.105:0/874524060 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47f008a3e0 con 0x7f47f80d43e0 2024-03-08T08:28:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.855+0000 7f47f659c640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f47d806b4d0 0x7f47d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.856+0000 7f47f659c640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f47d806b4d0 0x7f47d806d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47e8036000 tx=0x7f47e803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:05.860+0000 7f47eeffd640 1 -- 172.21.15.105:0/874524060 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47f808d9c0 con 0x7f47f80d43e0 2024-03-08T08:28:06.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.029+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47f80c9b90 con 0x7f47f80d43e0 2024-03-08T08:28:06.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.029+0000 7f47eeffd640 1 -- 172.21.15.105:0/874524060 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47f80c9b90 con 0x7f47f80d43e0 2024-03-08T08:28:06.031 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:06.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f47d806b4d0 msgr2=0x7f47d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f47d806b4d0 0x7f47d806d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47e8036000 tx=0x7f47e803d040 comp rx=0 tx=0).stop 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d43e0 msgr2=0x7f47f80ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d43e0 0x7f47f80ca3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c780 tx=0x7f47f000cc40 comp rx=0 tx=0).stop 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 shutdown_connections 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f47d806b4d0 0x7f47d806d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47f80d43e0 0x7f47f80ca3f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47f80d2910 0x7f47f80c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 --2- 172.21.15.105:0/874524060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47f80d2530 0x7f47f80b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 >> 172.21.15.105:0/874524060 conn(0x7f47f809cfb0 msgr2=0x7f47f806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 shutdown_connections 2024-03-08T08:28:06.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.031+0000 7f47fcbac640 1 -- 172.21.15.105:0/874524060 wait complete. 2024-03-08T08:28:06.043 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.143+0000 7f856a177640 1 Processor -- start 2024-03-08T08:28:06.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.143+0000 7f856a177640 1 -- start start 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.143+0000 7f856a177640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415dfc0 0x7f85640b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.143+0000 7f856a177640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f856406c780 0x7f856406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.143+0000 7f856a177640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85640b24c0 con 0x7f85640aec20 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.143+0000 7f856a177640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f856406d120 con 0x7f856406c780 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.143+0000 7f856a177640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85640b3370 con 0x7f856415dfc0 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f85637fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415dfc0 0x7f85640b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f8562ffd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f856406c780 0x7f856406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f8562ffd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f856406c780 0x7f856406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:50592/0 (socket says 172.21.15.105:50592) 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f85637fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415dfc0 0x7f85640b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:48976/0 (socket says 172.21.15.105:48976) 2024-03-08T08:28:06.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f8563fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f85640aec20 0x7f85640af0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:57600/0 (socket says 172.21.15.105:57600) 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f8562ffd640 1 -- 172.21.15.105:0/3428695099 learned_addr learned my addr 172.21.15.105:0/3428695099 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f8562ffd640 1 -- 172.21.15.105:0/3428695099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415dfc0 msgr2=0x7f85640b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f8562ffd640 1 --2- 172.21.15.105:0/3428695099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415dfc0 0x7f85640b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f8562ffd640 1 -- 172.21.15.105:0/3428695099 >> v1:172.21.15.144:6789/0 conn(0x7f85640aec20 legacy=0x7f85640af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.144+0000 7f8562ffd640 1 -- 172.21.15.105:0/3428695099 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85640b2760 con 0x7f856406c780 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.145+0000 7f8562ffd640 1 --2- 172.21.15.105:0/3428695099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856406c780 0x7f856406cbe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8558009870 tx=0x7f855802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75dd16c0b48f17ef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.145+0000 7f85627fc640 1 -- 172.21.15.105:0/3428695099 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558034070 con 0x7f856406c780 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.145+0000 7f85627fc640 1 -- 172.21.15.105:0/3428695099 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8558030840 con 0x7f856406c780 2024-03-08T08:28:06.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.145+0000 7f85627fc640 1 -- 172.21.15.105:0/3428695099 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558030c10 con 0x7f856406c780 2024-03-08T08:28:06.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.145+0000 7f856a177640 1 -- 172.21.15.105:0/3428695099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856406c780 msgr2=0x7f856406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.145+0000 7f856a177640 1 --2- 172.21.15.105:0/3428695099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856406c780 0x7f856406cbe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8558009870 tx=0x7f855802b980 comp rx=0 tx=0).stop 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.146+0000 7f856a177640 1 -- 172.21.15.105:0/3428695099 shutdown_connections 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.146+0000 7f856a177640 1 --2- 172.21.15.105:0/3428695099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f856406c780 0x7f856406cbe0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.146+0000 7f856a177640 1 --2- 172.21.15.105:0/3428695099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856415dfc0 0x7f85640b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.146+0000 7f856a177640 1 -- 172.21.15.105:0/3428695099 >> 172.21.15.105:0/3428695099 conn(0x7f856409aa00 msgr2=0x7f856409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.146+0000 7f856a177640 1 -- 172.21.15.105:0/3428695099 shutdown_connections 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.146+0000 7f856a177640 1 -- 172.21.15.105:0/3428695099 wait complete. 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.147+0000 7f856a177640 1 Processor -- start 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.147+0000 7f856a177640 1 -- start start 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f856a177640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856406c780 0x7f856414e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f856a177640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640aec20 0x7f85641609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f856a177640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f856415dfc0 0x7f85641515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f856a177640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640b3000 con 0x7f856415dfc0 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f856a177640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640b2d20 con 0x7f85640aec20 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f856a177640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856415e3a0 con 0x7f856406c780 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f856415dfc0 0x7f85641515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f856415dfc0 0x7f85641515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59190/0 (socket says 172.21.15.105:59190) 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 -- 172.21.15.105:0/3648980801 learned_addr learned my addr 172.21.15.105:0/3648980801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8562ffd640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640aec20 0x7f85641609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f85637fe640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856406c780 0x7f856414e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 -- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856406c780 msgr2=0x7f856414e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856406c780 0x7f856414e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 -- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640aec20 msgr2=0x7f85641609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640aec20 0x7f85641609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 -- 172.21.15.105:0/3648980801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8564157700 con 0x7f856415dfc0 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f85637fe640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856406c780 0x7f856414e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f8563fff640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f856415dfc0 0x7f85641515b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f855c00c780 tx=0x7f855c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f854bfff640 1 -- 172.21.15.105:0/3648980801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c017070 con 0x7f856415dfc0 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f854bfff640 1 -- 172.21.15.105:0/3648980801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c00a640 con 0x7f856415dfc0 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f854bfff640 1 -- 172.21.15.105:0/3648980801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c00aa30 con 0x7f856415dfc0 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85641579c0 con 0x7f856415dfc0 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.148+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856414f180 con 0x7f856415dfc0 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.149+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856414efa0 con 0x7f856415dfc0 2024-03-08T08:28:06.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.150+0000 7f854bfff640 1 -- 172.21.15.105:0/3648980801 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f855c01a070 con 0x7f856415dfc0 2024-03-08T08:28:06.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.150+0000 7f854bfff640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.150+0000 7f854bfff640 1 -- 172.21.15.105:0/3648980801 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f855c089740 con 0x7f856415dfc0 2024-03-08T08:28:06.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.150+0000 7f85637fe640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.151+0000 7f85637fe640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f855006b4d0 0x7f855006d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8554009af0 tx=0x7f8554008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.154+0000 7f854bfff640 1 -- 172.21.15.105:0/3648980801 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f856414efa0 con 0x7f856415dfc0 2024-03-08T08:28:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.326+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8564067cb0 con 0x7f856415dfc0 2024-03-08T08:28:06.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.327+0000 7f854bfff640 1 -- 172.21.15.105:0/3648980801 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f8564067cb0 con 0x7f856415dfc0 2024-03-08T08:28:06.328 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:06.328 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:06.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f855006b4d0 msgr2=0x7f855006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f855006b4d0 0x7f855006d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8554009af0 tx=0x7f8554008040 comp rx=0 tx=0).stop 2024-03-08T08:28:06.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f856415dfc0 msgr2=0x7f85641515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f856415dfc0 0x7f85641515b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f855c00c780 tx=0x7f855c00cc40 comp rx=0 tx=0).stop 2024-03-08T08:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 shutdown_connections 2024-03-08T08:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f856415dfc0 0x7f85641515b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85640aec20 0x7f85641609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 --2- 172.21.15.105:0/3648980801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f856406c780 0x7f856414e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 >> 172.21.15.105:0/3648980801 conn(0x7f856409aa00 msgr2=0x7f85640b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 shutdown_connections 2024-03-08T08:28:06.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.329+0000 7f856a177640 1 -- 172.21.15.105:0/3648980801 wait complete. 2024-03-08T08:28:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:06.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:06.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.443+0000 7f83bada1640 1 Processor -- start 2024-03-08T08:28:06.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.443+0000 7f83bada1640 1 -- start start 2024-03-08T08:28:06.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83bada1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83b4161470 0x7f83b406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83bada1640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f83b4161850 0x7f83b40d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83bada1640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b40d48a0 con 0x7f83b406d370 2024-03-08T08:28:06.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83bada1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b40d2ba0 con 0x7f83b4161470 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83bada1640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b40d4b80 con 0x7f83b4161850 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83b8b3d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83b4161470 0x7f83b406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83b8b3d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83b4161470 0x7f83b406c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59202/0 (socket says 172.21.15.105:59202) 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83b8b3d640 1 -- 172.21.15.105:0/901556951 learned_addr learned my addr 172.21.15.105:0/901556951 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.444+0000 7f83b3fff640 1 --2- 172.21.15.105:0/901556951 >> v2:172.21.15.144:3300/0 conn(0x7f83b4161850 0x7f83b40d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.445+0000 7f83b8b3d640 1 -- 172.21.15.105:0/901556951 >> v2:172.21.15.144:3300/0 conn(0x7f83b4161850 msgr2=0x7f83b40d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.445+0000 7f83b8b3d640 1 --2- 172.21.15.105:0/901556951 >> v2:172.21.15.144:3300/0 conn(0x7f83b4161850 0x7f83b40d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.445+0000 7f83b8b3d640 1 -- 172.21.15.105:0/901556951 >> v1:172.21.15.144:6789/0 conn(0x7f83b406d370 legacy=0x7f83b406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.445+0000 7f83b8b3d640 1 -- 172.21.15.105:0/901556951 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b406c170 con 0x7f83b4161470 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.445+0000 7f83b8b3d640 1 --2- 172.21.15.105:0/901556951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b4161470 0x7f83b406c830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f83b408d5e0 tx=0x7f83a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166e2029564ae128 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.445+0000 7f83b37fe640 1 -- 172.21.15.105:0/901556951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4035020 con 0x7f83b4161470 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83b37fe640 1 -- 172.21.15.105:0/901556951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a4031870 con 0x7f83b4161470 2024-03-08T08:28:06.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83b37fe640 1 -- 172.21.15.105:0/901556951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4031c40 con 0x7f83b4161470 2024-03-08T08:28:06.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83bada1640 1 -- 172.21.15.105:0/901556951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b4161470 msgr2=0x7f83b406c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83bada1640 1 --2- 172.21.15.105:0/901556951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b4161470 0x7f83b406c830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f83b408d5e0 tx=0x7f83a402b910 comp rx=0 tx=0).stop 2024-03-08T08:28:06.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83bada1640 1 -- 172.21.15.105:0/901556951 shutdown_connections 2024-03-08T08:28:06.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83bada1640 1 --2- 172.21.15.105:0/901556951 >> v2:172.21.15.144:3300/0 conn(0x7f83b4161850 0x7f83b40d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83bada1640 1 --2- 172.21.15.105:0/901556951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b4161470 0x7f83b406c830 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83bada1640 1 -- 172.21.15.105:0/901556951 >> 172.21.15.105:0/901556951 conn(0x7f83b40d3ce0 msgr2=0x7f83b40d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:06.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83bada1640 1 -- 172.21.15.105:0/901556951 shutdown_connections 2024-03-08T08:28:06.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.446+0000 7f83bada1640 1 -- 172.21.15.105:0/901556951 wait complete. 2024-03-08T08:28:06.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.447+0000 7f83bada1640 1 Processor -- start 2024-03-08T08:28:06.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.447+0000 7f83bada1640 1 -- start start 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83bada1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b406d370 0x7f83b40a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83bada1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b4161470 0x7f83b415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83bada1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b4161850 0x7f83b40cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83bada1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b406cf40 con 0x7f83b406d370 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83bada1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b406bcb0 con 0x7f83b4161850 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83bada1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40ce0b0 con 0x7f83b4161470 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b8b3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b406d370 0x7f83b40a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b8b3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b406d370 0x7f83b40a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59204/0 (socket says 172.21.15.105:59204) 2024-03-08T08:28:06.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b8b3d640 1 -- 172.21.15.105:0/1319967137 learned_addr learned my addr 172.21.15.105:0/1319967137 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b8b3d640 1 -- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b4161470 msgr2=0x7f83b415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b3fff640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b4161470 0x7f83b415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b933e640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b4161850 0x7f83b40cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b8b3d640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b4161470 0x7f83b415ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b8b3d640 1 -- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b4161850 msgr2=0x7f83b40cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b8b3d640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b4161850 0x7f83b40cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.448+0000 7f83b8b3d640 1 -- 172.21.15.105:0/1319967137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40c77a0 con 0x7f83b406d370 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.449+0000 7f83b933e640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b4161850 0x7f83b40cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.449+0000 7f83b8b3d640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b406d370 0x7f83b40a16b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f83a402b950 tx=0x7f83a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.449+0000 7f83b17fa640 1 -- 172.21.15.105:0/1319967137 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4040070 con 0x7f83b406d370 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.449+0000 7f83b17fa640 1 -- 172.21.15.105:0/1319967137 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a4030d00 con 0x7f83b406d370 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.449+0000 7f83b17fa640 1 -- 172.21.15.105:0/1319967137 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4035040 con 0x7f83b406d370 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.449+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b40c7a80 con 0x7f83b406d370 2024-03-08T08:28:06.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.449+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b40c8040 con 0x7f83b406d370 2024-03-08T08:28:06.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.450+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b4075120 con 0x7f83b406d370 2024-03-08T08:28:06.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.450+0000 7f83b17fa640 1 -- 172.21.15.105:0/1319967137 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83a404b060 con 0x7f83b406d370 2024-03-08T08:28:06.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.450+0000 7f83b17fa640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f83a006b4d0 0x7f83a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.451+0000 7f83b3fff640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f83a006b4d0 0x7f83a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.451+0000 7f83b17fa640 1 -- 172.21.15.105:0/1319967137 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83a40abed0 con 0x7f83b406d370 2024-03-08T08:28:06.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.451+0000 7f83b3fff640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f83a006b4d0 0x7f83a006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f83b41f6aa0 tx=0x7f83a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.455+0000 7f83b17fa640 1 -- 172.21.15.105:0/1319967137 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83a407ade0 con 0x7f83b406d370 2024-03-08T08:28:06.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.624+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83b4074600 con 0x7f83b406d370 2024-03-08T08:28:06.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.625+0000 7f83b17fa640 1 -- 172.21.15.105:0/1319967137 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83a407a6e0 con 0x7f83b406d370 2024-03-08T08:28:06.626 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f83a006b4d0 msgr2=0x7f83a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f83a006b4d0 0x7f83a006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f83b41f6aa0 tx=0x7f83a8008040 comp rx=0 tx=0).stop 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b406d370 msgr2=0x7f83b40a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b406d370 0x7f83b40a16b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f83a402b950 tx=0x7f83a40046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 shutdown_connections 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f83a006b4d0 0x7f83a006d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83b4161850 0x7f83b40cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83b4161470 0x7f83b415ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 --2- 172.21.15.105:0/1319967137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83b406d370 0x7f83b40a16b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 >> 172.21.15.105:0/1319967137 conn(0x7f83b40d3ce0 msgr2=0x7f83b40b8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 shutdown_connections 2024-03-08T08:28:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.627+0000 7f83bada1640 1 -- 172.21.15.105:0/1319967137 wait complete. 2024-03-08T08:28:06.639 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:06.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.739+0000 7fe190de9640 1 Processor -- start 2024-03-08T08:28:06.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.739+0000 7fe190de9640 1 -- start start 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.740+0000 7fe190de9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe18c15bea0 0x7fe18c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.740+0000 7fe190de9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18c15df30 0x7fe18c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.740+0000 7fe190de9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c15f7b0 con 0x7fe18c0b5a60 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.740+0000 7fe190de9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18c15e710 con 0x7fe18c15bea0 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.740+0000 7fe190de9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15e9f0 con 0x7fe18c15df30 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18ad9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe18c0b5a60 0x7fe18c09baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41888/0 (socket says 172.21.15.105:41888) 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1247166298 learned_addr learned my addr 172.21.15.105:0/1247166298 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18a59c640 1 --2- 172.21.15.105:0/1247166298 >> v2:172.21.15.144:3300/0 conn(0x7fe18c15bea0 0x7fe18c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe189d9b640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18c15df30 0x7fe18c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481622141 0 0) 0x7fe18c15f7b0 con 0x7fe18c0b5a60 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe174003520 con 0x7fe18c0b5a60 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 842500250 0 0) 0x7fe174003520 con 0x7fe18c0b5a60 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18c15df30 msgr2=0x7fe18c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18c15df30 0x7fe18c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 >> v2:172.21.15.144:3300/0 conn(0x7fe18c15bea0 msgr2=0x7fe18c0b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 --2- 172.21.15.105:0/1247166298 >> v2:172.21.15.144:3300/0 conn(0x7fe18c15bea0 0x7fe18c0b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c15ecd0 con 0x7fe18c0b5a60 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fe184001670 con 0x7fe18c0b5a60 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 >> v1:172.21.15.105:6789/0 conn(0x7fe18c0b5a60 legacy=0x7fe18c09baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18c15df30 0x7fe1740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174003f30 0x7fe174004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe174004d40 0x7fe1740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15e710 con 0x7fe174004d40 2024-03-08T08:28:06.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15e9f0 con 0x7fe174003f30 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17400b9c0 con 0x7fe18c15df30 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18ad9d640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe174004d40 0x7fe1740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe18a59c640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18c15df30 0x7fe1740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.741+0000 7fe189d9b640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174003f30 0x7fe174004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18c15df30 msgr2=0x7fe1740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18ad9d640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe18c15df30 0x7fe1740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174003f30 msgr2=0x7fe174004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18ad9d640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe174003f30 0x7fe174004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1247166298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174003520 con 0x7fe174004d40 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18ad9d640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe174004d40 0x7fe1740051c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe184001050 tx=0x7fe174005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d32744f0220309 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184007d70 con 0x7fe174004d40 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184030070 con 0x7fe174004d40 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.742+0000 7fe18959a640 1 -- 172.21.15.105:0/1247166298 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18402cd70 con 0x7fe174004d40 2024-03-08T08:28:06.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.743+0000 7fe190de9640 1 -- 172.21.15.105:0/1247166298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe174004d40 msgr2=0x7fe1740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.743+0000 7fe190de9640 1 --2- 172.21.15.105:0/1247166298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe174004d40 0x7fe1740051c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe184001050 tx=0x7fe174005830 comp rx=0 tx=0).stop 2024-03-08T08:28:06.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.743+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1247166298 reap_dead start 2024-03-08T08:28:06.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.743+0000 7fe190de9640 1 -- 172.21.15.105:0/1247166298 shutdown_connections 2024-03-08T08:28:06.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.743+0000 7fe190de9640 1 -- 172.21.15.105:0/1247166298 >> 172.21.15.105:0/1247166298 conn(0x7fe18c09aa00 msgr2=0x7fe18c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:06.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.743+0000 7fe190de9640 1 -- 172.21.15.105:0/1247166298 shutdown_connections 2024-03-08T08:28:06.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.743+0000 7fe190de9640 1 -- 172.21.15.105:0/1247166298 wait complete. 2024-03-08T08:28:06.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.744+0000 7fe190de9640 1 Processor -- start 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.744+0000 7fe190de9640 1 -- start start 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe190de9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe174004d40 0x7fe18c14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe190de9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe18c1f2790 0x7fe18c140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe190de9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18c14e010 0x7fe18c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe190de9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15d290 con 0x7fe18c14e010 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe190de9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15f290 con 0x7fe18c1f2790 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe190de9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15e3b0 con 0x7fe174004d40 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18c14e010 0x7fe18c1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18c14e010 0x7fe18c1533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59228/0 (socket says 172.21.15.105:59228) 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1450564014 learned_addr learned my addr 172.21.15.105:0/1450564014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18a59c640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe174004d40 0x7fe18c14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe189d9b640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe18c1f2790 0x7fe18c140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe174004d40 msgr2=0x7fe18c14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18ad9d640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe174004d40 0x7fe18c14faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe18c1f2790 msgr2=0x7fe18c140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18ad9d640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe18c1f2790 0x7fe18c140580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe18ad9d640 1 -- 172.21.15.105:0/1450564014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c14e840 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.745+0000 7fe189d9b640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe18c1f2790 0x7fe18c140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.746+0000 7fe18ad9d640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18c14e010 0x7fe18c1533b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe184000c00 tx=0x7fe184001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.746+0000 7fe17affd640 1 -- 172.21.15.105:0/1450564014 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184004bf0 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.746+0000 7fe17affd640 1 -- 172.21.15.105:0/1450564014 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184004d90 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.746+0000 7fe17affd640 1 -- 172.21.15.105:0/1450564014 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184007270 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.746+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c14eb20 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.746+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c151090 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.747+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c151290 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.747+0000 7fe17affd640 1 -- 172.21.15.105:0/1450564014 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe184007410 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.748+0000 7fe17affd640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b500 0x7fe16c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.748+0000 7fe17affd640 1 -- 172.21.15.105:0/1450564014 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1840ab290 con 0x7fe18c14e010 2024-03-08T08:28:06.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.748+0000 7fe18a59c640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b500 0x7fe16c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:06.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.748+0000 7fe18a59c640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b500 0x7fe16c06d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe17c014510 tx=0x7fe17c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:06.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.752+0000 7fe17affd640 1 -- 172.21.15.105:0/1450564014 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe18c151290 con 0x7fe18c14e010 2024-03-08T08:28:06.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.921+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe18c075120 con 0x7fe18c14e010 2024-03-08T08:28:06.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.922+0000 7fe17affd640 1 -- 172.21.15.105:0/1450564014 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe18407a110 con 0x7fe18c14e010 2024-03-08T08:28:06.923 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.923+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b500 msgr2=0x7fe16c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.923+0000 7fe190de9640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b500 0x7fe16c06d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe17c014510 tx=0x7fe17c00b040 comp rx=0 tx=0).stop 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.923+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18c14e010 msgr2=0x7fe18c1533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.923+0000 7fe190de9640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18c14e010 0x7fe18c1533b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe184000c00 tx=0x7fe184001a40 comp rx=0 tx=0).stop 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.923+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 shutdown_connections 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.923+0000 7fe190de9640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe16c06b500 0x7fe16c06d9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.924+0000 7fe190de9640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe18c14e010 0x7fe18c1533b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.924+0000 7fe190de9640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe18c1f2790 0x7fe18c140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.924+0000 7fe190de9640 1 --2- 172.21.15.105:0/1450564014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe174004d40 0x7fe18c14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.924+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 >> 172.21.15.105:0/1450564014 conn(0x7fe18c09aa00 msgr2=0x7fe18c06fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:06.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.924+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 shutdown_connections 2024-03-08T08:28:06.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:06.924+0000 7fe190de9640 1 -- 172.21.15.105:0/1450564014 wait complete. 2024-03-08T08:28:06.935 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:07.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.036+0000 7feab8d2c640 1 Processor -- start 2024-03-08T08:28:07.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.037+0000 7feab8d2c640 1 -- start start 2024-03-08T08:28:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.037+0000 7feab8d2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feab4066db0 0x7feab40b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.037+0000 7feab8d2c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab40cd900 con 0x7feab406ac80 2024-03-08T08:28:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.037+0000 7feab8d2c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab40b5160 con 0x7feab40af760 2024-03-08T08:28:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.037+0000 7feab8d2c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab40b57b0 con 0x7feab4066db0 2024-03-08T08:28:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.038+0000 7feab1d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feab4066db0 0x7feab40b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.038+0000 7feab259c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feab406ac80 0x7feab406b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41900/0 (socket says 172.21.15.105:41900) 2024-03-08T08:28:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.038+0000 7feab259c640 1 -- 172.21.15.105:0/776088115 learned_addr learned my addr 172.21.15.105:0/776088115 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.038+0000 7feab1d9b640 1 -- 172.21.15.105:0/776088115 >> v1:172.21.15.144:6789/0 conn(0x7feab40af760 legacy=0x7feab415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.038+0000 7feab1d9b640 1 -- 172.21.15.105:0/776088115 >> v1:172.21.15.105:6789/0 conn(0x7feab406ac80 legacy=0x7feab406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.038+0000 7feab1d9b640 1 -- 172.21.15.105:0/776088115 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40b5440 con 0x7feab4066db0 2024-03-08T08:28:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.038+0000 7feab159a640 1 -- 172.21.15.105:0/776088115 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303260608 0 0) 0x7feab40cd900 con 0x7feab406ac80 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab1d9b640 1 --2- 172.21.15.105:0/776088115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab4066db0 0x7feab40b5c40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feaa80097d0 tx=0x7feaa802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0350734320706ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab159a640 1 -- 172.21.15.105:0/776088115 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa80315a0 con 0x7feab4066db0 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab159a640 1 -- 172.21.15.105:0/776088115 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa8037070 con 0x7feab4066db0 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab159a640 1 -- 172.21.15.105:0/776088115 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8031960 con 0x7feab4066db0 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab8d2c640 1 -- 172.21.15.105:0/776088115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab4066db0 msgr2=0x7feab40b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab8d2c640 1 --2- 172.21.15.105:0/776088115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab4066db0 0x7feab40b5c40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feaa80097d0 tx=0x7feaa802b840 comp rx=0 tx=0).stop 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab8d2c640 1 -- 172.21.15.105:0/776088115 shutdown_connections 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab8d2c640 1 --2- 172.21.15.105:0/776088115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab4066db0 0x7feab40b5c40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab8d2c640 1 -- 172.21.15.105:0/776088115 >> 172.21.15.105:0/776088115 conn(0x7feab4072110 msgr2=0x7feab4072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab8d2c640 1 -- 172.21.15.105:0/776088115 shutdown_connections 2024-03-08T08:28:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.039+0000 7feab8d2c640 1 -- 172.21.15.105:0/776088115 wait complete. 2024-03-08T08:28:07.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.041+0000 7feab8d2c640 1 Processor -- start 2024-03-08T08:28:07.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.041+0000 7feab8d2c640 1 -- start start 2024-03-08T08:28:07.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.041+0000 7feab8d2c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab4066db0 0x7feab41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.041+0000 7feab8d2c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab406ac80 0x7feab41f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.041+0000 7feab8d2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40af760 0x7feab41f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.041+0000 7feab8d2c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab415cbb0 con 0x7feab40af760 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.041+0000 7feab8d2c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab415ca30 con 0x7feab406ac80 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.041+0000 7feab8d2c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab415cd30 con 0x7feab4066db0 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40af760 0x7feab41f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40af760 0x7feab41f17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59248/0 (socket says 172.21.15.105:59248) 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 -- 172.21.15.105:0/780485348 learned_addr learned my addr 172.21.15.105:0/780485348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab1d9b640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab406ac80 0x7feab41f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab259c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab4066db0 0x7feab41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 -- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab4066db0 msgr2=0x7feab41588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab4066db0 0x7feab41588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 -- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab406ac80 msgr2=0x7feab41f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab406ac80 0x7feab41f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 -- 172.21.15.105:0/780485348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab41f5db0 con 0x7feab40af760 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab259c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab4066db0 0x7feab41588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab1d9b640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab406ac80 0x7feab41f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:07.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab2d9d640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40af760 0x7feab41f17d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feaac00b410 tx=0x7feaac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feaa2ffd640 1 -- 172.21.15.105:0/780485348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac015030 con 0x7feab40af760 2024-03-08T08:28:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feaa2ffd640 1 -- 172.21.15.105:0/780485348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac004020 con 0x7feab40af760 2024-03-08T08:28:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab41f6090 con 0x7feab40af760 2024-03-08T08:28:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feaa2ffd640 1 -- 172.21.15.105:0/780485348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac004450 con 0x7feab40af760 2024-03-08T08:28:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.042+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab40c3d50 con 0x7feab40af760 2024-03-08T08:28:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.043+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab4075120 con 0x7feab40af760 2024-03-08T08:28:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.043+0000 7feaa2ffd640 1 -- 172.21.15.105:0/780485348 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feaac018070 con 0x7feab40af760 2024-03-08T08:28:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.044+0000 7feaa2ffd640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fea9406b4d0 0x7fea9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.044+0000 7feaa2ffd640 1 -- 172.21.15.105:0/780485348 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feaac089c70 con 0x7feab40af760 2024-03-08T08:28:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.044+0000 7feab259c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fea9406b4d0 0x7fea9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.044+0000 7feab259c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fea9406b4d0 0x7fea9406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feaa40012e0 tx=0x7feaa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.048+0000 7feaa2ffd640 1 -- 172.21.15.105:0/780485348 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feaac058b80 con 0x7feab40af760 2024-03-08T08:28:07.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.218+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feab40b61a0 con 0x7feab40af760 2024-03-08T08:28:07.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.218+0000 7feaa2ffd640 1 -- 172.21.15.105:0/780485348 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7feaac058480 con 0x7feab40af760 2024-03-08T08:28:07.220 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fea9406b4d0 msgr2=0x7fea9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fea9406b4d0 0x7fea9406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feaa40012e0 tx=0x7feaa4009040 comp rx=0 tx=0).stop 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40af760 msgr2=0x7feab41f17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40af760 0x7feab41f17d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feaac00b410 tx=0x7feaac00b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 shutdown_connections 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fea9406b4d0 0x7fea9406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feab40af760 0x7feab41f17d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab406ac80 0x7feab41f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 --2- 172.21.15.105:0/780485348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab4066db0 0x7feab41588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 >> 172.21.15.105:0/780485348 conn(0x7feab4072110 msgr2=0x7feab40cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 shutdown_connections 2024-03-08T08:28:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.220+0000 7feab8d2c640 1 -- 172.21.15.105:0/780485348 wait complete. 2024-03-08T08:28:07.232 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:07.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.331+0000 7f2c6c5d0640 1 Processor -- start 2024-03-08T08:28:07.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.331+0000 7f2c6c5d0640 1 -- start start 2024-03-08T08:28:07.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.332+0000 7f2c6c5d0640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c64161850 0x7f2c6406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.332+0000 7f2c6c5d0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c640d48a0 con 0x7f2c6406bcb0 2024-03-08T08:28:07.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.332+0000 7f2c6c5d0640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c640d4b80 con 0x7f2c64161470 2024-03-08T08:28:07.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.332+0000 7f2c6c5d0640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6406c170 con 0x7f2c64161850 2024-03-08T08:28:07.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c69b6b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2c6406bcb0 0x7f2c640b2450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41914/0 (socket says 172.21.15.105:41914) 2024-03-08T08:28:07.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c69b6b640 1 -- 172.21.15.105:0/3360141163 learned_addr learned my addr 172.21.15.105:0/3360141163 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308937256 0 0) 0x7f2c640d48a0 con 0x7f2c6406bcb0 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6ab6d640 1 --2- 172.21.15.105:0/3360141163 >> v2:172.21.15.144:3300/0 conn(0x7f2c64161850 0x7f2c6406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c50003520 con 0x7f2c6406bcb0 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1029947604 0 0) 0x7f2c50003520 con 0x7f2c6406bcb0 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 >> v2:172.21.15.144:3300/0 conn(0x7f2c64161850 msgr2=0x7f2c6406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 --2- 172.21.15.105:0/3360141163 >> v2:172.21.15.144:3300/0 conn(0x7f2c64161850 0x7f2c6406c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 >> v1:172.21.15.144:6789/0 conn(0x7f2c64161470 legacy=0x7f2c6406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6406c450 con 0x7f2c6406bcb0 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f2c60001670 con 0x7f2c6406bcb0 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 >> v1:172.21.15.105:6789/0 conn(0x7f2c6406bcb0 legacy=0x7f2c640b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c500039b0 0x7f2c50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c50004800 0x7f2c50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c500051c0 0x7f2c5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6406c170 con 0x7f2c500051c0 2024-03-08T08:28:07.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5000c160 con 0x7f2c500039b0 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5000c360 con 0x7f2c50004800 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6ab6d640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c500051c0 0x7f2c5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6a36c640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c500039b0 0x7f2c50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c69b6b640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c50004800 0x7f2c50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6ab6d640 1 -- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c50004800 msgr2=0x7f2c50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6ab6d640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c50004800 0x7f2c50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6ab6d640 1 -- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c500039b0 msgr2=0x7f2c50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6ab6d640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c500039b0 0x7f2c50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c6ab6d640 1 -- 172.21.15.105:0/3360141163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50003520 con 0x7f2c500051c0 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.333+0000 7f2c69b6b640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c50004800 0x7f2c50004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.334+0000 7f2c6ab6d640 1 -- 172.21.15.105:0/3360141163 reap_dead start 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.334+0000 7f2c6ab6d640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c500051c0 0x7f2c5000b9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009fc0 tx=0x7f2c50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2d0e88f3770fa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.334+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c009c10 con 0x7f2c500051c0 2024-03-08T08:28:07.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.334+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c02cc60 con 0x7f2c500051c0 2024-03-08T08:28:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.334+0000 7f2c6936a640 1 -- 172.21.15.105:0/3360141163 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c003bd0 con 0x7f2c500051c0 2024-03-08T08:28:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.335+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3360141163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c500051c0 msgr2=0x7f2c5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.335+0000 7f2c6c5d0640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c500051c0 0x7f2c5000b9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009fc0 tx=0x7f2c50005830 comp rx=0 tx=0).stop 2024-03-08T08:28:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.335+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3360141163 shutdown_connections 2024-03-08T08:28:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.335+0000 7f2c6c5d0640 1 --2- 172.21.15.105:0/3360141163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c500051c0 0x7f2c5000b9c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.335+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3360141163 >> 172.21.15.105:0/3360141163 conn(0x7f2c640d3ce0 msgr2=0x7f2c640d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.335+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3360141163 shutdown_connections 2024-03-08T08:28:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.335+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3360141163 wait complete. 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.336+0000 7f2c6c5d0640 1 Processor -- start 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.336+0000 7f2c6c5d0640 1 -- start start 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6c5d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c500051c0 0x7f2c640c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6c5d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c640cd530 0x7f2c6415d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6c5d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c641588e0 0x7f2c640c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6c5d0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6406d4f0 con 0x7f2c641588e0 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6c5d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6406d370 con 0x7f2c640cd530 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6c5d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6406d670 con 0x7f2c500051c0 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c641588e0 0x7f2c640c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c641588e0 0x7f2c640c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59272/0 (socket says 172.21.15.105:59272) 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 -- 172.21.15.105:0/3913988017 learned_addr learned my addr 172.21.15.105:0/3913988017 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c69b6b640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c640cd530 0x7f2c6415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6a36c640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c500051c0 0x7f2c640c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 -- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c500051c0 msgr2=0x7f2c640c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c500051c0 0x7f2c640c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 -- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c640cd530 msgr2=0x7f2c6415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c640cd530 0x7f2c6415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 -- 172.21.15.105:0/3913988017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640cadb0 con 0x7f2c641588e0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c69b6b640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c640cd530 0x7f2c6415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6ab6d640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c641588e0 0x7f2c640c9790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c02c6d0 tx=0x7f2c5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c5affd640 1 -- 172.21.15.105:0/3913988017 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c003990 con 0x7f2c641588e0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c5affd640 1 -- 172.21.15.105:0/3913988017 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c044070 con 0x7f2c641588e0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6415d8a0 con 0x7f2c641588e0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.337+0000 7f2c5affd640 1 -- 172.21.15.105:0/3913988017 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c003d90 con 0x7f2c641588e0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.338+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c640cb350 con 0x7f2c641588e0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.338+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6415ede0 con 0x7f2c641588e0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.339+0000 7f2c5affd640 1 -- 172.21.15.105:0/3913988017 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c5c005d10 con 0x7f2c641588e0 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.339+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c4806b4d0 0x7f2c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.339+0000 7f2c5affd640 1 -- 172.21.15.105:0/3913988017 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c5c0ac930 con 0x7f2c641588e0 2024-03-08T08:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.339+0000 7f2c6a36c640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c4806b4d0 0x7f2c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.340+0000 7f2c6a36c640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c4806b4d0 0x7f2c4806d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.343+0000 7f2c5affd640 1 -- 172.21.15.105:0/3913988017 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c6415ede0 con 0x7f2c641588e0 2024-03-08T08:28:07.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.515+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c64075120 con 0x7f2c641588e0 2024-03-08T08:28:07.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.516+0000 7f2c5affd640 1 -- 172.21.15.105:0/3913988017 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f2c5c07b7b0 con 0x7f2c641588e0 2024-03-08T08:28:07.517 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:07.517 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:07.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c4806b4d0 msgr2=0x7f2c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c4806b4d0 0x7f2c4806d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c54009040 comp rx=0 tx=0).stop 2024-03-08T08:28:07.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c641588e0 msgr2=0x7f2c640c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c641588e0 0x7f2c640c9790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c02c6d0 tx=0x7f2c5c009750 comp rx=0 tx=0).stop 2024-03-08T08:28:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 shutdown_connections 2024-03-08T08:28:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c4806b4d0 0x7f2c4806d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c641588e0 0x7f2c640c9790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c640cd530 0x7f2c6415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 --2- 172.21.15.105:0/3913988017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c500051c0 0x7f2c640c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 >> 172.21.15.105:0/3913988017 conn(0x7f2c640d3ce0 msgr2=0x7f2c6415f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 shutdown_connections 2024-03-08T08:28:07.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.518+0000 7f2c6c5d0640 1 -- 172.21.15.105:0/3913988017 wait complete. 2024-03-08T08:28:07.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:07.530 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:07.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.632+0000 7fcb29d25640 1 Processor -- start 2024-03-08T08:28:07.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.632+0000 7fcb29d25640 1 -- start start 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb29d25640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d43e0 0x7fcb240b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb29d25640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb240d2910 0x7fcb2415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb29d25640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb240d48f0 con 0x7fcb240d2530 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb29d25640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb240d4bb0 con 0x7fcb240d2910 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb29d25640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240ceb30 con 0x7fcb240d43e0 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb23fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb240d2910 0x7fcb2415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb23fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcb240d2910 0x7fcb2415de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59282/0 (socket says 172.21.15.105:59282) 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb23fff640 1 -- 172.21.15.105:0/1444792036 learned_addr learned my addr 172.21.15.105:0/1444792036 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb237fe640 1 --1- 172.21.15.105:0/1444792036 >> v1:172.21.15.105:6789/0 conn(0x7fcb240d2530 0x7fcb240b3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41920/0 (socket says 172.21.15.105:41920) 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb22ffd640 1 --2- 172.21.15.105:0/1444792036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d43e0 0x7fcb240b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb23fff640 1 -- 172.21.15.105:0/1444792036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d43e0 msgr2=0x7fcb240b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:07.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb23fff640 1 --2- 172.21.15.105:0/1444792036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d43e0 0x7fcb240b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb23fff640 1 -- 172.21.15.105:0/1444792036 >> v1:172.21.15.105:6789/0 conn(0x7fcb240d2530 legacy=0x7fcb240b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb23fff640 1 -- 172.21.15.105:0/1444792036 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240cee10 con 0x7fcb240d2910 2024-03-08T08:28:07.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.633+0000 7fcb227fc640 1 -- 172.21.15.105:0/1444792036 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575313175 0 0) 0x7fcb240d48f0 con 0x7fcb240d2530 2024-03-08T08:28:07.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb23fff640 1 --2- 172.21.15.105:0/1444792036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d2910 0x7fcb2415de00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009e50 tx=0x7fcb1c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4c0437e02332ff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb227fc640 1 -- 172.21.15.105:0/1444792036 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c02c9e0 con 0x7fcb240d2910 2024-03-08T08:28:07.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb227fc640 1 -- 172.21.15.105:0/1444792036 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c02cb80 con 0x7fcb240d2910 2024-03-08T08:28:07.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb227fc640 1 -- 172.21.15.105:0/1444792036 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0343f0 con 0x7fcb240d2910 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb29d25640 1 -- 172.21.15.105:0/1444792036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d2910 msgr2=0x7fcb2415de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1444792036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d2910 0x7fcb2415de00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009e50 tx=0x7fcb1c02be40 comp rx=0 tx=0).stop 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb29d25640 1 -- 172.21.15.105:0/1444792036 shutdown_connections 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1444792036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d2910 0x7fcb2415de00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1444792036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d43e0 0x7fcb240b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.634+0000 7fcb29d25640 1 -- 172.21.15.105:0/1444792036 >> 172.21.15.105:0/1444792036 conn(0x7fcb2409cfb0 msgr2=0x7fcb240d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.635+0000 7fcb29d25640 1 -- 172.21.15.105:0/1444792036 shutdown_connections 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.635+0000 7fcb29d25640 1 -- 172.21.15.105:0/1444792036 wait complete. 2024-03-08T08:28:07.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.636+0000 7fcb29d25640 1 Processor -- start 2024-03-08T08:28:07.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.636+0000 7fcb29d25640 1 -- start start 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.636+0000 7fcb29d25640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d2530 0x7fcb24154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb29d25640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb240d2910 0x7fcb2415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb29d25640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d43e0 0x7fcb2415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb29d25640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2415e340 con 0x7fcb240d43e0 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb29d25640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240b3080 con 0x7fcb240d2910 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb29d25640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2415e4c0 con 0x7fcb240d2530 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d43e0 0x7fcb2415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d43e0 0x7fcb2415b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59286/0 (socket says 172.21.15.105:59286) 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 -- 172.21.15.105:0/1705926066 learned_addr learned my addr 172.21.15.105:0/1705926066 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb22ffd640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb240d2910 0x7fcb2415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb237fe640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d2530 0x7fcb24154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 -- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d2530 msgr2=0x7fcb24154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d2530 0x7fcb24154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 -- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb240d2910 msgr2=0x7fcb2415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb240d2910 0x7fcb2415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 -- 172.21.15.105:0/1705926066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240c9b00 con 0x7fcb240d43e0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb22ffd640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb240d2910 0x7fcb2415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb237fe640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d2530 0x7fcb24154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb23fff640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d43e0 0x7fcb2415b4a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009e50 tx=0x7fcb1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb0bfff640 1 -- 172.21.15.105:0/1705926066 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c043030 con 0x7fcb240d43e0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb0bfff640 1 -- 172.21.15.105:0/1705926066 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c03e450 con 0x7fcb240d43e0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb240c9de0 con 0x7fcb240d43e0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb0bfff640 1 -- 172.21.15.105:0/1705926066 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c03e620 con 0x7fcb240d43e0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.637+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb240ca3a0 con 0x7fcb240d43e0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.638+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb24075120 con 0x7fcb240d43e0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.639+0000 7fcb0bfff640 1 -- 172.21.15.105:0/1705926066 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb1c046070 con 0x7fcb240d43e0 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.639+0000 7fcb0bfff640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb1006b500 0x7fcb1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.639+0000 7fcb0bfff640 1 -- 172.21.15.105:0/1705926066 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb1c0ac780 con 0x7fcb240d43e0 2024-03-08T08:28:07.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.639+0000 7fcb237fe640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb1006b500 0x7fcb1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.639+0000 7fcb237fe640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb1006b500 0x7fcb1006d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb14000f60 tx=0x7fcb14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.643+0000 7fcb0bfff640 1 -- 172.21.15.105:0/1705926066 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb1c04d070 con 0x7fcb240d43e0 2024-03-08T08:28:07.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.814+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb2408dd60 con 0x7fcb240d43e0 2024-03-08T08:28:07.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.815+0000 7fcb0bfff640 1 -- 172.21.15.105:0/1705926066 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb1c07b7b0 con 0x7fcb240d43e0 2024-03-08T08:28:07.816 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.816+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb1006b500 msgr2=0x7fcb1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.816+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb1006b500 0x7fcb1006d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcb14000f60 tx=0x7fcb14009040 comp rx=0 tx=0).stop 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.816+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d43e0 msgr2=0x7fcb2415b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.816+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d43e0 0x7fcb2415b4a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009e50 tx=0x7fcb1c004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.817+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 shutdown_connections 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.817+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb1006b500 0x7fcb1006d9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.817+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb240d43e0 0x7fcb2415b4a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.817+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb240d2910 0x7fcb2415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.817+0000 7fcb29d25640 1 --2- 172.21.15.105:0/1705926066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb240d2530 0x7fcb24154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.817+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 >> 172.21.15.105:0/1705926066 conn(0x7fcb2409cfb0 msgr2=0x7fcb2406c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:07.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.817+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 shutdown_connections 2024-03-08T08:28:07.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.817+0000 7fcb29d25640 1 -- 172.21.15.105:0/1705926066 wait complete. 2024-03-08T08:28:07.828 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:07.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.930+0000 7fa50d6b5640 1 Processor -- start 2024-03-08T08:28:07.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.930+0000 7fa50d6b5640 1 -- start start 2024-03-08T08:28:07.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa50d6b5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa50815df20 0x7fa50806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa50d6b5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa50806b390 0x7fa5080ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa50d6b5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50806f830 con 0x7fa5080b5990 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa50d6b5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50806dbb0 con 0x7fa50815df20 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa50d6b5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50806fad0 con 0x7fa50806b390 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa506ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa50815df20 0x7fa50806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa506ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa50815df20 0x7fa50806c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59296/0 (socket says 172.21.15.105:59296) 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa506ffd640 1 -- 172.21.15.105:0/4124088235 learned_addr learned my addr 172.21.15.105:0/4124088235 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.931+0000 7fa5067fc640 1 --1- 172.21.15.105:0/4124088235 >> v1:172.21.15.105:6789/0 conn(0x7fa5080b5990 0x7fa50806ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41924/0 (socket says 172.21.15.105:41924) 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.932+0000 7fa5077fe640 1 --2- 172.21.15.105:0/4124088235 >> v2:172.21.15.144:3300/0 conn(0x7fa50806b390 0x7fa5080ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.932+0000 7fa506ffd640 1 -- 172.21.15.105:0/4124088235 >> v2:172.21.15.144:3300/0 conn(0x7fa50806b390 msgr2=0x7fa5080ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.932+0000 7fa506ffd640 1 --2- 172.21.15.105:0/4124088235 >> v2:172.21.15.144:3300/0 conn(0x7fa50806b390 0x7fa5080ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.932+0000 7fa506ffd640 1 -- 172.21.15.105:0/4124088235 >> v1:172.21.15.105:6789/0 conn(0x7fa5080b5990 legacy=0x7fa50806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.932+0000 7fa506ffd640 1 -- 172.21.15.105:0/4124088235 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50806fdb0 con 0x7fa50815df20 2024-03-08T08:28:07.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.932+0000 7fa506ffd640 1 --2- 172.21.15.105:0/4124088235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50815df20 0x7fa50806c8b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5080b5e20 tx=0x7fa4f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459417a764ea993d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.932+0000 7fa505ffb640 1 -- 172.21.15.105:0/4124088235 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f802ca80 con 0x7fa50815df20 2024-03-08T08:28:07.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa505ffb640 1 -- 172.21.15.105:0/4124088235 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f802cc20 con 0x7fa50815df20 2024-03-08T08:28:07.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa505ffb640 1 -- 172.21.15.105:0/4124088235 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80343f0 con 0x7fa50815df20 2024-03-08T08:28:07.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa50d6b5640 1 -- 172.21.15.105:0/4124088235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50815df20 msgr2=0x7fa50806c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/4124088235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50815df20 0x7fa50806c8b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5080b5e20 tx=0x7fa4f802bee0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa50d6b5640 1 -- 172.21.15.105:0/4124088235 shutdown_connections 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/4124088235 >> v2:172.21.15.144:3300/0 conn(0x7fa50806b390 0x7fa5080ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/4124088235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50815df20 0x7fa50806c8b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa50d6b5640 1 -- 172.21.15.105:0/4124088235 >> 172.21.15.105:0/4124088235 conn(0x7fa50806bd90 msgr2=0x7fa50806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa50d6b5640 1 -- 172.21.15.105:0/4124088235 shutdown_connections 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.933+0000 7fa50d6b5640 1 -- 172.21.15.105:0/4124088235 wait complete. 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.934+0000 7fa50d6b5640 1 Processor -- start 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.934+0000 7fa50d6b5640 1 -- start start 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa50d6b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50806b390 0x7fa5081f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa50d6b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5080b5990 0x7fa5081609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa50d6b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa50815df20 0x7fa508156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa50d6b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080adeb0 con 0x7fa50806b390 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa50d6b5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50806e0e0 con 0x7fa50815df20 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa50d6b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5080ae030 con 0x7fa5080b5990 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa506ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50806b390 0x7fa5081f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa506ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50806b390 0x7fa5081f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59306/0 (socket says 172.21.15.105:59306) 2024-03-08T08:28:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa506ffd640 1 -- 172.21.15.105:0/3894397973 learned_addr learned my addr 172.21.15.105:0/3894397973 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa5077fe640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa50815df20 0x7fa508156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa5067fc640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5080b5990 0x7fa5081609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa506ffd640 1 -- 172.21.15.105:0/3894397973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5080b5990 msgr2=0x7fa5081609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa506ffd640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5080b5990 0x7fa5081609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa506ffd640 1 -- 172.21.15.105:0/3894397973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa50815df20 msgr2=0x7fa508156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.935+0000 7fa506ffd640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa50815df20 0x7fa508156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.936+0000 7fa506ffd640 1 -- 172.21.15.105:0/3894397973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50815ed10 con 0x7fa50806b390 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.936+0000 7fa506ffd640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50806b390 0x7fa5081f6750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8036040 tx=0x7fa4f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.936+0000 7fa4ef7fe640 1 -- 172.21.15.105:0/3894397973 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8043020 con 0x7fa50806b390 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.936+0000 7fa4ef7fe640 1 -- 172.21.15.105:0/3894397973 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f803e450 con 0x7fa50806b390 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.936+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50815f050 con 0x7fa50806b390 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.936+0000 7fa4ef7fe640 1 -- 172.21.15.105:0/3894397973 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8009510 con 0x7fa50806b390 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.936+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50815f890 con 0x7fa50806b390 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.937+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa508075120 con 0x7fa50806b390 2024-03-08T08:28:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.937+0000 7fa4ef7fe640 1 -- 172.21.15.105:0/3894397973 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4f8046020 con 0x7fa50806b390 2024-03-08T08:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.937+0000 7fa4ef7fe640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa4f406b4d0 0x7fa4f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.938+0000 7fa4ef7fe640 1 -- 172.21.15.105:0/3894397973 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4f80ac930 con 0x7fa50806b390 2024-03-08T08:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.938+0000 7fa5067fc640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa4f406b4d0 0x7fa4f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.938+0000 7fa5067fc640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa4f406b4d0 0x7fa4f406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc007fd0 tx=0x7fa4fc007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:07.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:07.942+0000 7fa4ef7fe640 1 -- 172.21.15.105:0/3894397973 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4f804d030 con 0x7fa50806b390 2024-03-08T08:28:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.116+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa50815b3d0 con 0x7fa50806b390 2024-03-08T08:28:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.116+0000 7fa4ef7fe640 1 -- 172.21.15.105:0/3894397973 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa50815f050 con 0x7fa50806b390 2024-03-08T08:28:08.117 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa4f406b4d0 msgr2=0x7fa4f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa4f406b4d0 0x7fa4f406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc007fd0 tx=0x7fa4fc007e00 comp rx=0 tx=0).stop 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50806b390 msgr2=0x7fa5081f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50806b390 0x7fa5081f6750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8036040 tx=0x7fa4f8004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 shutdown_connections 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa4f406b4d0 0x7fa4f406d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa50815df20 0x7fa508156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5080b5990 0x7fa5081609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 --2- 172.21.15.105:0/3894397973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa50806b390 0x7fa5081f6750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 >> 172.21.15.105:0/3894397973 conn(0x7fa50806bd90 msgr2=0x7fa50806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 shutdown_connections 2024-03-08T08:28:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.118+0000 7fa50d6b5640 1 -- 172.21.15.105:0/3894397973 wait complete. 2024-03-08T08:28:08.130 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:08.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.231+0000 7f9b08a2e640 1 Processor -- start 2024-03-08T08:28:08.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.231+0000 7f9b08a2e640 1 -- start start 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.232+0000 7f9b08a2e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b0406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.232+0000 7f9b08a2e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b0406b390 0x7f9b040ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.232+0000 7f9b08a2e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0406f830 con 0x7f9b040b5990 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.232+0000 7f9b08a2e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0406dbb0 con 0x7f9b0406b390 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.232+0000 7f9b08a2e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0406fad0 con 0x7f9b0415df20 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b0259c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b0406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b02d9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b0406b390 0x7f9b040ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b01d9b640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9b040b5990 0x7f9b0406ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:57634/0 (socket says 172.21.15.105:57634) 2024-03-08T08:28:08.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b01d9b640 1 -- 172.21.15.105:0/1413897297 learned_addr learned my addr 172.21.15.105:0/1413897297 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b02d9d640 1 -- 172.21.15.105:0/1413897297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 msgr2=0x7f9b0406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b02d9d640 1 --2- 172.21.15.105:0/1413897297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b0406c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b02d9d640 1 -- 172.21.15.105:0/1413897297 >> v1:172.21.15.144:6789/0 conn(0x7f9b040b5990 legacy=0x7f9b0406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b02d9d640 1 -- 172.21.15.105:0/1413897297 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0406fdb0 con 0x7f9b0406b390 2024-03-08T08:28:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.233+0000 7f9b0259c640 1 --2- 172.21.15.105:0/1413897297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b0406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.234+0000 7f9b02d9d640 1 --2- 172.21.15.105:0/1413897297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406b390 0x7f9b040ad970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0097a0 tx=0x7f9afc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89fed178c8db8f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.234+0000 7f9b0159a640 1 -- 172.21.15.105:0/1413897297 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0315a0 con 0x7f9b0406b390 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.234+0000 7f9b0159a640 1 -- 172.21.15.105:0/1413897297 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc037070 con 0x7f9b0406b390 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.234+0000 7f9b0159a640 1 -- 172.21.15.105:0/1413897297 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc031960 con 0x7f9b0406b390 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.234+0000 7f9b08a2e640 1 -- 172.21.15.105:0/1413897297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406b390 msgr2=0x7f9b040ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.234+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/1413897297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406b390 0x7f9b040ad970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0097a0 tx=0x7f9afc02b840 comp rx=0 tx=0).stop 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.235+0000 7f9b08a2e640 1 -- 172.21.15.105:0/1413897297 shutdown_connections 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.235+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/1413897297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b0406b390 0x7f9b040ad970 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.235+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/1413897297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b0406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.235+0000 7f9b08a2e640 1 -- 172.21.15.105:0/1413897297 >> 172.21.15.105:0/1413897297 conn(0x7f9b0406bd90 msgr2=0x7f9b0406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:08.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.235+0000 7f9b08a2e640 1 -- 172.21.15.105:0/1413897297 shutdown_connections 2024-03-08T08:28:08.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.235+0000 7f9b08a2e640 1 -- 172.21.15.105:0/1413897297 wait complete. 2024-03-08T08:28:08.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.236+0000 7f9b08a2e640 1 Processor -- start 2024-03-08T08:28:08.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.236+0000 7f9b08a2e640 1 -- start start 2024-03-08T08:28:08.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.236+0000 7f9b08a2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0406b390 0x7f9b04156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.236+0000 7f9b08a2e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b040b5990 0x7f9b04154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.236+0000 7f9b08a2e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b040ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b08a2e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040adeb0 con 0x7f9b0406b390 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b08a2e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0406e0e0 con 0x7f9b040b5990 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b08a2e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040ae030 con 0x7f9b0415df20 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0406b390 0x7f9b04156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0406b390 0x7f9b04156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59324/0 (socket says 172.21.15.105:59324) 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 -- 172.21.15.105:0/2446329498 learned_addr learned my addr 172.21.15.105:0/2446329498 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b02d9d640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b040ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b01d9b640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b040b5990 0x7f9b04154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 -- 172.21.15.105:0/2446329498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 msgr2=0x7f9b040ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b040ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 -- 172.21.15.105:0/2446329498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b040b5990 msgr2=0x7f9b04154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b040b5990 0x7f9b04154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 -- 172.21.15.105:0/2446329498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b04155190 con 0x7f9b0406b390 2024-03-08T08:28:08.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b0259c640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0406b390 0x7f9b04156f00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009ec0 tx=0x7f9af400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9af2ffd640 1 -- 172.21.15.105:0/2446329498 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4018020 con 0x7f9b0406b390 2024-03-08T08:28:08.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9af2ffd640 1 -- 172.21.15.105:0/2446329498 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4014680 con 0x7f9b0406b390 2024-03-08T08:28:08.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9af2ffd640 1 -- 172.21.15.105:0/2446329498 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4014a50 con 0x7f9b0406b390 2024-03-08T08:28:08.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b04143cd0 con 0x7f9b0406b390 2024-03-08T08:28:08.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.237+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b04144290 con 0x7f9b0406b390 2024-03-08T08:28:08.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.238+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b04075120 con 0x7f9b0406b390 2024-03-08T08:28:08.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.239+0000 7f9af2ffd640 1 -- 172.21.15.105:0/2446329498 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9af4005d10 con 0x7f9b0406b390 2024-03-08T08:28:08.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.239+0000 7f9af2ffd640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9ae406b4d0 0x7f9ae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.239+0000 7f9af2ffd640 1 -- 172.21.15.105:0/2446329498 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9af408b020 con 0x7f9b0406b390 2024-03-08T08:28:08.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.239+0000 7f9b01d9b640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9ae406b4d0 0x7f9ae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.240+0000 7f9b01d9b640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9ae406b4d0 0x7f9ae406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b0406f2f0 tx=0x7f9af8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.243+0000 7f9af2ffd640 1 -- 172.21.15.105:0/2446329498 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9af4022030 con 0x7f9b0406b390 2024-03-08T08:28:08.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.412+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b040b1100 con 0x7f9b0406b390 2024-03-08T08:28:08.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.412+0000 7f9af2ffd640 1 -- 172.21.15.105:0/2446329498 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9af4010040 con 0x7f9b0406b390 2024-03-08T08:28:08.413 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:08.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9ae406b4d0 msgr2=0x7f9ae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9ae406b4d0 0x7f9ae406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b0406f2f0 tx=0x7f9af8009040 comp rx=0 tx=0).stop 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0406b390 msgr2=0x7f9b04156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0406b390 0x7f9b04156f00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009ec0 tx=0x7f9af400c490 comp rx=0 tx=0).stop 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 shutdown_connections 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9ae406b4d0 0x7f9ae406d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0415df20 0x7f9b040ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b040b5990 0x7f9b04154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 --2- 172.21.15.105:0/2446329498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0406b390 0x7f9b04156f00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 >> 172.21.15.105:0/2446329498 conn(0x7f9b0406bd90 msgr2=0x7f9b0406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 shutdown_connections 2024-03-08T08:28:08.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.414+0000 7f9b08a2e640 1 -- 172.21.15.105:0/2446329498 wait complete. 2024-03-08T08:28:08.449 INFO:tasks.cephfs_test_runner:Starting test: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) That the per-client labelled perf counters depict the clients 2024-03-08T08:28:08.449 INFO:tasks.cephfs_test_runner:test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T08:28:08.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:08.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.548+0000 7f5d57547640 1 Processor -- start 2024-03-08T08:28:08.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.548+0000 7f5d57547640 1 -- start start 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d57547640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5d500ce330 0x7f5d500ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d57547640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500b29e0 con 0x7f5d500cfb40 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d57547640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500d1650 con 0x7f5d50161520 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d57547640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d500d1930 con 0x7f5d500ce330 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d54ae2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5d500ce330 0x7f5d500ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d54ae2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5d500ce330 0x7f5d500ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59336/0 (socket says 172.21.15.105:59336) 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d54ae2640 1 -- 172.21.15.105:0/150929301 learned_addr learned my addr 172.21.15.105:0/150929301 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d55ae4640 1 --1- 172.21.15.105:0/150929301 >> v1:172.21.15.105:6789/0 conn(0x7f5d500cfb40 0x7f5d500cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41926/0 (socket says 172.21.15.105:41926) 2024-03-08T08:28:08.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d54ae2640 1 -- 172.21.15.105:0/150929301 >> v1:172.21.15.144:6789/0 conn(0x7f5d50161520 legacy=0x7f5d500b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d54ae2640 1 -- 172.21.15.105:0/150929301 >> v1:172.21.15.105:6789/0 conn(0x7f5d500cfb40 legacy=0x7f5d500cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.549+0000 7f5d54ae2640 1 -- 172.21.15.105:0/150929301 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d50161900 con 0x7f5d500ce330 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.550+0000 7f5d54ae2640 1 --2- 172.21.15.105:0/150929301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d500ce7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009870 tx=0x7f5d4002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdbe9c68032d831 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.550+0000 7f5d47fff640 1 -- 172.21.15.105:0/150929301 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4002f840 con 0x7f5d500ce330 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.550+0000 7f5d47fff640 1 -- 172.21.15.105:0/150929301 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4002f9e0 con 0x7f5d500ce330 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.550+0000 7f5d47fff640 1 -- 172.21.15.105:0/150929301 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4002fdb0 con 0x7f5d500ce330 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.551+0000 7f5d57547640 1 -- 172.21.15.105:0/150929301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 msgr2=0x7f5d500ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.551+0000 7f5d57547640 1 --2- 172.21.15.105:0/150929301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d500ce7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009870 tx=0x7f5d4002ba20 comp rx=0 tx=0).stop 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.551+0000 7f5d57547640 1 -- 172.21.15.105:0/150929301 shutdown_connections 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.551+0000 7f5d57547640 1 --2- 172.21.15.105:0/150929301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d500ce7c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.551+0000 7f5d57547640 1 -- 172.21.15.105:0/150929301 >> 172.21.15.105:0/150929301 conn(0x7f5d500b36a0 msgr2=0x7f5d500b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:08.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.551+0000 7f5d57547640 1 -- 172.21.15.105:0/150929301 shutdown_connections 2024-03-08T08:28:08.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.551+0000 7f5d57547640 1 -- 172.21.15.105:0/150929301 wait complete. 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.552+0000 7f5d57547640 1 Processor -- start 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.552+0000 7f5d57547640 1 -- start start 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.552+0000 7f5d57547640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d5014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d57547640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d500cfb40 0x7f5d500b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d57547640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d50161520 0x7f5d500c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d57547640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5006ca40 con 0x7f5d500ce330 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d57547640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5006c770 con 0x7f5d50161520 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d57547640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500b43d0 con 0x7f5d500cfb40 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d5014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d5014ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59348/0 (socket says 172.21.15.105:59348) 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 -- 172.21.15.105:0/3310810599 learned_addr learned my addr 172.21.15.105:0/3310810599 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d55ae4640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d50161520 0x7f5d500c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 -- 172.21.15.105:0/3310810599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d500cfb40 msgr2=0x7f5d500b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d500cfb40 0x7f5d500b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 -- 172.21.15.105:0/3310810599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d50161520 msgr2=0x7f5d500c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d50161520 0x7f5d500c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 -- 172.21.15.105:0/3310810599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500c6b20 con 0x7f5d500ce330 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d552e3640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d5014ee90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00c3c0 tx=0x7f5d4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d45ffb640 1 -- 172.21.15.105:0/3310810599 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0052d0 con 0x7f5d500ce330 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d45ffb640 1 -- 172.21.15.105:0/3310810599 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c005470 con 0x7f5d500ce330 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d500c6e00 con 0x7f5d500ce330 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d45ffb640 1 -- 172.21.15.105:0/3310810599 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c007260 con 0x7f5d500ce330 2024-03-08T08:28:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.553+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d500c73c0 con 0x7f5d500ce330 2024-03-08T08:28:08.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.555+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d50075120 con 0x7f5d500ce330 2024-03-08T08:28:08.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.555+0000 7f5d45ffb640 1 -- 172.21.15.105:0/3310810599 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d4c007400 con 0x7f5d500ce330 2024-03-08T08:28:08.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.555+0000 7f5d45ffb640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5d3406b4d0 0x7f5d3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.555+0000 7f5d54ae2640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5d3406b4d0 0x7f5d3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.555+0000 7f5d45ffb640 1 -- 172.21.15.105:0/3310810599 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d4c089720 con 0x7f5d500ce330 2024-03-08T08:28:08.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.556+0000 7f5d54ae2640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5d3406b4d0 0x7f5d3406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d40036000 tx=0x7f5d40039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.560+0000 7f5d45ffb640 1 -- 172.21.15.105:0/3310810599 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d4c0585a0 con 0x7f5d500ce330 2024-03-08T08:28:08.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.736+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d500cdde0 con 0x7f5d500ce330 2024-03-08T08:28:08.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.737+0000 7f5d45ffb640 1 -- 172.21.15.105:0/3310810599 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f5d4c057ea0 con 0x7f5d500ce330 2024-03-08T08:28:08.738 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:08.738 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5d3406b4d0 msgr2=0x7f5d3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5d3406b4d0 0x7f5d3406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d40036000 tx=0x7f5d40039040 comp rx=0 tx=0).stop 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 msgr2=0x7f5d5014ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d5014ee90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00c3c0 tx=0x7f5d4c00c880 comp rx=0 tx=0).stop 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 shutdown_connections 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5d3406b4d0 0x7f5d3406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d50161520 0x7f5d500c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d500cfb40 0x7f5d500b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 --2- 172.21.15.105:0/3310810599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d500ce330 0x7f5d5014ee90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 >> 172.21.15.105:0/3310810599 conn(0x7f5d500b36a0 msgr2=0x7f5d5006fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:08.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 shutdown_connections 2024-03-08T08:28:08.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.739+0000 7f5d57547640 1 -- 172.21.15.105:0/3310810599 wait complete. 2024-03-08T08:28:08.751 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:08.751 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:08.845 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:28:08.848 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:28:08.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.853+0000 7f97aa64e640 1 Processor -- start 2024-03-08T08:28:08.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.853+0000 7f97aa64e640 1 -- start start 2024-03-08T08:28:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.854+0000 7f97aa64e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f97a40b4e40 0x7f97a40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.854+0000 7f97aa64e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b5a60 0x7f97a415c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.854+0000 7f97aa64e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a415f7b0 con 0x7f97a415bea0 2024-03-08T08:28:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.854+0000 7f97aa64e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a415e710 con 0x7f97a40b4e40 2024-03-08T08:28:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.854+0000 7f97aa64e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a415e9f0 con 0x7f97a40b5a60 2024-03-08T08:28:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.854+0000 7f97a8beb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b5a60 0x7f97a415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a37fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f97a40b4e40 0x7f97a40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a3fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f97a415bea0 0x7f97a415df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:57654/0 (socket says 172.21.15.105:57654) 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a8beb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b5a60 0x7f97a415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:49142/0 (socket says 172.21.15.105:49142) 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a3fff640 1 -- 172.21.15.105:0/2438579361 learned_addr learned my addr 172.21.15.105:0/2438579361 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a2ffd640 1 -- 172.21.15.105:0/2438579361 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745535981 0 0) 0x7f97a415f7b0 con 0x7f97a415bea0 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a2ffd640 1 -- 172.21.15.105:0/2438579361 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f978c003520 con 0x7f97a415bea0 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a37fe640 1 -- 172.21.15.105:0/2438579361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b5a60 msgr2=0x7f97a415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a37fe640 1 --2- 172.21.15.105:0/2438579361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b5a60 0x7f97a415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a37fe640 1 -- 172.21.15.105:0/2438579361 >> v1:172.21.15.144:6789/0 conn(0x7f97a415bea0 legacy=0x7f97a415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a37fe640 1 -- 172.21.15.105:0/2438579361 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a415ecd0 con 0x7f97a40b4e40 2024-03-08T08:28:08.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.855+0000 7f97a8beb640 1 --2- 172.21.15.105:0/2438579361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b5a60 0x7f97a415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:08.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.856+0000 7f97a37fe640 1 --2- 172.21.15.105:0/2438579361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a40b4e40 0x7f97a40b5520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f97980097a0 tx=0x7f979802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524e73deeff9f67c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.856+0000 7f97a2ffd640 1 -- 172.21.15.105:0/2438579361 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798035020 con 0x7f97a40b4e40 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.856+0000 7f97a2ffd640 1 -- 172.21.15.105:0/2438579361 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9798031830 con 0x7f97a40b4e40 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.856+0000 7f97a2ffd640 1 -- 172.21.15.105:0/2438579361 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798031c00 con 0x7f97a40b4e40 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.857+0000 7f97aa64e640 1 -- 172.21.15.105:0/2438579361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a40b4e40 msgr2=0x7f97a40b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.857+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2438579361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a40b4e40 0x7f97a40b5520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f97980097a0 tx=0x7f979802b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.857+0000 7f97aa64e640 1 -- 172.21.15.105:0/2438579361 shutdown_connections 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.857+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2438579361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b5a60 0x7f97a415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.857+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2438579361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a40b4e40 0x7f97a40b5520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.857+0000 7f97aa64e640 1 -- 172.21.15.105:0/2438579361 >> 172.21.15.105:0/2438579361 conn(0x7f97a409aa00 msgr2=0x7f97a409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.857+0000 7f97aa64e640 1 -- 172.21.15.105:0/2438579361 shutdown_connections 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.857+0000 7f97aa64e640 1 -- 172.21.15.105:0/2438579361 wait complete. 2024-03-08T08:28:08.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.858+0000 7f97aa64e640 1 Processor -- start 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.858+0000 7f97aa64e640 1 -- start start 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97aa64e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b4e40 0x7f97a41439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97aa64e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a40b5a60 0x7f97a40b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97aa64e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a415bea0 0x7f97a40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97aa64e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a409cb00 con 0x7f97a40b5a60 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97aa64e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a415f290 con 0x7f97a415bea0 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97aa64e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a415c7c0 con 0x7f97a40b4e40 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a40b5a60 0x7f97a40b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a40b5a60 0x7f97a40b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59350/0 (socket says 172.21.15.105:59350) 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a8beb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a415bea0 0x7f97a40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a37fe640 1 -- 172.21.15.105:0/2647699832 learned_addr learned my addr 172.21.15.105:0/2647699832 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a3fff640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b4e40 0x7f97a41439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a8beb640 1 -- 172.21.15.105:0/2647699832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b4e40 msgr2=0x7f97a41439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a8beb640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b4e40 0x7f97a41439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a8beb640 1 -- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a40b5a60 msgr2=0x7f97a40b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a8beb640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a40b5a60 0x7f97a40b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a8beb640 1 -- 172.21.15.105:0/2647699832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a415a570 con 0x7f97a415bea0 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.859+0000 7f97a37fe640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a40b5a60 0x7f97a40b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.860+0000 7f97a8beb640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a415bea0 0x7f97a40b28f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f979c00efc0 tx=0x7f979c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.860+0000 7f97a0ff9640 1 -- 172.21.15.105:0/2647699832 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c018030 con 0x7f97a415bea0 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.860+0000 7f97a0ff9640 1 -- 172.21.15.105:0/2647699832 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00cd30 con 0x7f97a415bea0 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.860+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a415a850 con 0x7f97a415bea0 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.860+0000 7f97a0ff9640 1 -- 172.21.15.105:0/2647699832 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c003bb0 con 0x7f97a415bea0 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.860+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a4159800 con 0x7f97a415bea0 2024-03-08T08:28:08.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.861+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a4075120 con 0x7f97a415bea0 2024-03-08T08:28:08.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.861+0000 7f97a0ff9640 1 -- 172.21.15.105:0/2647699832 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f979c005d10 con 0x7f97a415bea0 2024-03-08T08:28:08.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.862+0000 7f97a0ff9640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f979006b500 0x7f979006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:08.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.862+0000 7f97a0ff9640 1 -- 172.21.15.105:0/2647699832 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f979c08b710 con 0x7f97a415bea0 2024-03-08T08:28:08.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.862+0000 7f97a3fff640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f979006b500 0x7f979006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:08.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.862+0000 7f97a3fff640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f979006b500 0x7f979006d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9794005880 tx=0x7f9794009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:08.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:08.866+0000 7f97a0ff9640 1 -- 172.21.15.105:0/2647699832 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f979c010040 con 0x7f97a415bea0 2024-03-08T08:28:09.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.040+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97a40b4530 con 0x7f97a415bea0 2024-03-08T08:28:09.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.040+0000 7f97a0ff9640 1 -- 172.21.15.105:0/2647699832 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f979c059f20 con 0x7f97a415bea0 2024-03-08T08:28:09.041 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:09.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f979006b500 msgr2=0x7f979006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f979006b500 0x7f979006d9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9794005880 tx=0x7f9794009040 comp rx=0 tx=0).stop 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a415bea0 msgr2=0x7f97a40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a415bea0 0x7f97a40b28f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f979c00efc0 tx=0x7f979c00c490 comp rx=0 tx=0).stop 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 shutdown_connections 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f979006b500 0x7f979006d9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a415bea0 0x7f97a40b28f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a40b5a60 0x7f97a40b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 --2- 172.21.15.105:0/2647699832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97a40b4e40 0x7f97a41439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.042+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 >> 172.21.15.105:0/2647699832 conn(0x7f97a409aa00 msgr2=0x7f97a415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.043+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 shutdown_connections 2024-03-08T08:28:09.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.043+0000 7f97aa64e640 1 -- 172.21.15.105:0/2647699832 wait complete. 2024-03-08T08:28:09.054 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:09.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.156+0000 7f5224181640 1 Processor -- start 2024-03-08T08:28:09.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.156+0000 7f5224181640 1 -- start start 2024-03-08T08:28:09.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5224181640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f521c15dd80 0x7f521c1623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5224181640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f521c0b2b90 0x7f521c0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5224181640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c162900 con 0x7f521c0b1550 2024-03-08T08:28:09.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5224181640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0b2700 con 0x7f521c15dd80 2024-03-08T08:28:09.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5224181640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0af190 con 0x7f521c0b2b90 2024-03-08T08:28:09.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5221f1d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f521c15dd80 0x7f521c1623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f522171c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f521c0b1550 0x7f521c0aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41932/0 (socket says 172.21.15.105:41932) 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f522171c640 1 -- 172.21.15.105:0/2029053430 learned_addr learned my addr 172.21.15.105:0/2029053430 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f522271e640 1 --2- 172.21.15.105:0/2029053430 >> v2:172.21.15.144:3300/0 conn(0x7f521c0b2b90 0x7f521c0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5221f1d640 1 -- 172.21.15.105:0/2029053430 >> v2:172.21.15.144:3300/0 conn(0x7f521c0b2b90 msgr2=0x7f521c0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5221f1d640 1 --2- 172.21.15.105:0/2029053430 >> v2:172.21.15.144:3300/0 conn(0x7f521c0b2b90 0x7f521c0b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5221f1d640 1 -- 172.21.15.105:0/2029053430 >> v1:172.21.15.105:6789/0 conn(0x7f521c0b1550 legacy=0x7f521c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5220f1b640 1 -- 172.21.15.105:0/2029053430 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331200138 0 0) 0x7f520c001110 con 0x7f521c0b1550 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.157+0000 7f5221f1d640 1 -- 172.21.15.105:0/2029053430 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c0b1af0 con 0x7f521c15dd80 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.158+0000 7f5221f1d640 1 --2- 172.21.15.105:0/2029053430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c15dd80 0x7f521c1623c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f52180097a0 tx=0x7f521802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0583856be5b37c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.158+0000 7f5220f1b640 1 -- 172.21.15.105:0/2029053430 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218035020 con 0x7f521c15dd80 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.158+0000 7f5220f1b640 1 -- 172.21.15.105:0/2029053430 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5218031830 con 0x7f521c15dd80 2024-03-08T08:28:09.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.158+0000 7f5220f1b640 1 -- 172.21.15.105:0/2029053430 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218031c00 con 0x7f521c15dd80 2024-03-08T08:28:09.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.159+0000 7f5224181640 1 -- 172.21.15.105:0/2029053430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c15dd80 msgr2=0x7f521c1623c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.159+0000 7f5224181640 1 --2- 172.21.15.105:0/2029053430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c15dd80 0x7f521c1623c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f52180097a0 tx=0x7f521802b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.159+0000 7f5224181640 1 -- 172.21.15.105:0/2029053430 shutdown_connections 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.159+0000 7f5224181640 1 --2- 172.21.15.105:0/2029053430 >> v2:172.21.15.144:3300/0 conn(0x7f521c0b2b90 0x7f521c0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.159+0000 7f5224181640 1 --2- 172.21.15.105:0/2029053430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c15dd80 0x7f521c1623c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.159+0000 7f5224181640 1 -- 172.21.15.105:0/2029053430 >> 172.21.15.105:0/2029053430 conn(0x7f521c0b37a0 msgr2=0x7f521c0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.159+0000 7f5224181640 1 -- 172.21.15.105:0/2029053430 shutdown_connections 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.159+0000 7f5224181640 1 -- 172.21.15.105:0/2029053430 wait complete. 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.160+0000 7f5224181640 1 Processor -- start 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.160+0000 7f5224181640 1 -- start start 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5224181640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f521c0b1550 0x7f521c14a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5224181640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c0b2b90 0x7f521c158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5224181640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521c15dd80 0x7f521c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5224181640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0b2390 con 0x7f521c0b2b90 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5224181640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0b20b0 con 0x7f521c0b1550 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5224181640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0b3520 con 0x7f521c15dd80 2024-03-08T08:28:09.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c0b2b90 0x7f521c158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c0b2b90 0x7f521c158e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59372/0 (socket says 172.21.15.105:59372) 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 -- 172.21.15.105:0/316996050 learned_addr learned my addr 172.21.15.105:0/316996050 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5221f1d640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f521c0b1550 0x7f521c14a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522271e640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521c15dd80 0x7f521c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 -- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521c15dd80 msgr2=0x7f521c159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521c15dd80 0x7f521c159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 -- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f521c0b1550 msgr2=0x7f521c14a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f521c0b1550 0x7f521c14a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 -- 172.21.15.105:0/316996050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c0cc250 con 0x7f521c0b2b90 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522271e640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521c15dd80 0x7f521c159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f522171c640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c0b2b90 0x7f521c158e50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f520c00c430 tx=0x7f520c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f52127fc640 1 -- 172.21.15.105:0/316996050 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c0074e0 con 0x7f521c0b2b90 2024-03-08T08:28:09.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f52127fc640 1 -- 172.21.15.105:0/316996050 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c0047e0 con 0x7f521c0b2b90 2024-03-08T08:28:09.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f52127fc640 1 -- 172.21.15.105:0/316996050 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c004bd0 con 0x7f521c0b2b90 2024-03-08T08:28:09.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c0cc530 con 0x7f521c0b2b90 2024-03-08T08:28:09.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.161+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c0ccaf0 con 0x7f521c0b2b90 2024-03-08T08:28:09.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.162+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c075120 con 0x7f521c0b2b90 2024-03-08T08:28:09.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.163+0000 7f52127fc640 1 -- 172.21.15.105:0/316996050 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f520c011070 con 0x7f521c0b2b90 2024-03-08T08:28:09.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.163+0000 7f52127fc640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f520006b500 0x7f520006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.163+0000 7f52127fc640 1 -- 172.21.15.105:0/316996050 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f520c089d40 con 0x7f521c0b2b90 2024-03-08T08:28:09.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.163+0000 7f5221f1d640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f520006b500 0x7f520006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.164+0000 7f5221f1d640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f520006b500 0x7f520006d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5218036000 tx=0x7f521803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.167+0000 7f52127fc640 1 -- 172.21.15.105:0/316996050 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f521c162900 con 0x7f521c0b2b90 2024-03-08T08:28:09.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.338+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f521c15aeb0 con 0x7f521c0b2b90 2024-03-08T08:28:09.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.338+0000 7f52127fc640 1 -- 172.21.15.105:0/316996050 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f520c058bc0 con 0x7f521c0b2b90 2024-03-08T08:28:09.340 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f520006b500 msgr2=0x7f520006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f520006b500 0x7f520006d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5218036000 tx=0x7f521803c040 comp rx=0 tx=0).stop 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c0b2b90 msgr2=0x7f521c158e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c0b2b90 0x7f521c158e50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f520c00c430 tx=0x7f520c00c8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 shutdown_connections 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f520006b500 0x7f520006d9c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f521c15dd80 0x7f521c159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f521c0b2b90 0x7f521c158e50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 --2- 172.21.15.105:0/316996050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f521c0b1550 0x7f521c14a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 >> 172.21.15.105:0/316996050 conn(0x7f521c0b37a0 msgr2=0x7f521c06f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 shutdown_connections 2024-03-08T08:28:09.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.340+0000 7f5224181640 1 -- 172.21.15.105:0/316996050 wait complete. 2024-03-08T08:28:09.352 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.453+0000 7f26ac76c640 1 Processor -- start 2024-03-08T08:28:09.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.453+0000 7f26ac76c640 1 -- start start 2024-03-08T08:28:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26ac76c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d2530 0x7f26a40b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26ac76c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26a40d43e0 0x7f26a40b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26ac76c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a40b3880 con 0x7f26a40d2910 2024-03-08T08:28:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26ac76c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a40d48f0 con 0x7f26a40d43e0 2024-03-08T08:28:09.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26ac76c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40d4bb0 con 0x7f26a40d2530 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26a9d07640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26a40d43e0 0x7f26a40b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26aad09640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f26a40d2910 0x7f26a415de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41940/0 (socket says 172.21.15.105:41940) 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26aad09640 1 -- 172.21.15.105:0/2929446519 learned_addr learned my addr 172.21.15.105:0/2929446519 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26aa508640 1 --2- 172.21.15.105:0/2929446519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d2530 0x7f26a40b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26a9d07640 1 -- 172.21.15.105:0/2929446519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d2530 msgr2=0x7f26a40b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26a9d07640 1 --2- 172.21.15.105:0/2929446519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d2530 0x7f26a40b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26a9d07640 1 -- 172.21.15.105:0/2929446519 >> v1:172.21.15.105:6789/0 conn(0x7f26a40d2910 legacy=0x7f26a415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26a9d07640 1 -- 172.21.15.105:0/2929446519 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40ceb30 con 0x7f26a40d43e0 2024-03-08T08:28:09.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.454+0000 7f26a9506640 1 -- 172.21.15.105:0/2929446519 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66056357 0 0) 0x7f26a40b3880 con 0x7f26a40d2910 2024-03-08T08:28:09.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.455+0000 7f26a9d07640 1 --2- 172.21.15.105:0/2929446519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d43e0 0x7f26a40b3340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26940097a0 tx=0x7f269402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a659d1420c34f58 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.455+0000 7f26a9506640 1 -- 172.21.15.105:0/2929446519 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694038070 con 0x7f26a40d43e0 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.455+0000 7f26a9506640 1 -- 172.21.15.105:0/2929446519 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2694031700 con 0x7f26a40d43e0 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.455+0000 7f26a9506640 1 -- 172.21.15.105:0/2929446519 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694031ad0 con 0x7f26a40d43e0 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.456+0000 7f26ac76c640 1 -- 172.21.15.105:0/2929446519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d43e0 msgr2=0x7f26a40b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.456+0000 7f26ac76c640 1 --2- 172.21.15.105:0/2929446519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d43e0 0x7f26a40b3340 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f26940097a0 tx=0x7f269402b7a0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.456+0000 7f26ac76c640 1 -- 172.21.15.105:0/2929446519 shutdown_connections 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.456+0000 7f26ac76c640 1 --2- 172.21.15.105:0/2929446519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d43e0 0x7f26a40b3340 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.456+0000 7f26ac76c640 1 --2- 172.21.15.105:0/2929446519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d2530 0x7f26a40b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.456+0000 7f26ac76c640 1 -- 172.21.15.105:0/2929446519 >> 172.21.15.105:0/2929446519 conn(0x7f26a409cfb0 msgr2=0x7f26a40d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.456+0000 7f26ac76c640 1 -- 172.21.15.105:0/2929446519 shutdown_connections 2024-03-08T08:28:09.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.456+0000 7f26ac76c640 1 -- 172.21.15.105:0/2929446519 wait complete. 2024-03-08T08:28:09.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.457+0000 7f26ac76c640 1 Processor -- start 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.457+0000 7f26ac76c640 1 -- start start 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26ac76c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a40d2530 0x7f26a40ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26ac76c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d2910 0x7f26a41f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26ac76c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d43e0 0x7f26a4160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26ac76c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40b3080 con 0x7f26a40d2910 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26ac76c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40cf0f0 con 0x7f26a40d2530 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26ac76c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a415e510 con 0x7f26a40d43e0 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d2910 0x7f26a41f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d2910 0x7f26a41f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59404/0 (socket says 172.21.15.105:59404) 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 -- 172.21.15.105:0/3334776405 learned_addr learned my addr 172.21.15.105:0/3334776405 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26aa508640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a40d2530 0x7f26a40ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26aad09640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d43e0 0x7f26a4160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 -- 172.21.15.105:0/3334776405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d43e0 msgr2=0x7f26a4160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d43e0 0x7f26a4160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 -- 172.21.15.105:0/3334776405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a40d2530 msgr2=0x7f26a40ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a40d2530 0x7f26a40ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 -- 172.21.15.105:0/3334776405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a41f4670 con 0x7f26a40d2910 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f26a9d07640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d2910 0x7f26a41f3f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26940098c0 tx=0x7f26940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.458+0000 7f269affd640 1 -- 172.21.15.105:0/3334776405 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694038040 con 0x7f26a40d2910 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.459+0000 7f269affd640 1 -- 172.21.15.105:0/3334776405 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269403c610 con 0x7f26a40d2910 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.459+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a41f4950 con 0x7f26a40d2910 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.459+0000 7f269affd640 1 -- 172.21.15.105:0/3334776405 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269403c9e0 con 0x7f26a40d2910 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.459+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a41612b0 con 0x7f26a40d2910 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.459+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a4075120 con 0x7f26a40d2910 2024-03-08T08:28:09.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.460+0000 7f269affd640 1 -- 172.21.15.105:0/3334776405 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2694045020 con 0x7f26a40d2910 2024-03-08T08:28:09.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.460+0000 7f269affd640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f268806b4d0 0x7f268806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.460+0000 7f26aa508640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f268806b4d0 0x7f268806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.460+0000 7f269affd640 1 -- 172.21.15.105:0/3334776405 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26940aafd0 con 0x7f26a40d2910 2024-03-08T08:28:09.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.461+0000 7f26aa508640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f268806b4d0 0x7f268806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26a41605b0 tx=0x7f269c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.465+0000 7f269affd640 1 -- 172.21.15.105:0/3334776405 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2694030070 con 0x7f26a40d2910 2024-03-08T08:28:09.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.635+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26a409d500 con 0x7f26a40d2910 2024-03-08T08:28:09.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.636+0000 7f269affd640 1 -- 172.21.15.105:0/3334776405 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2694079890 con 0x7f26a40d2910 2024-03-08T08:28:09.637 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f268806b4d0 msgr2=0x7f268806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f268806b4d0 0x7f268806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26a41605b0 tx=0x7f269c009190 comp rx=0 tx=0).stop 2024-03-08T08:28:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d2910 msgr2=0x7f26a41f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d2910 0x7f26a41f3f00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26940098c0 tx=0x7f26940046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 shutdown_connections 2024-03-08T08:28:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f268806b4d0 0x7f268806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a40d43e0 0x7f26a4160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26a40d2910 0x7f26a41f3f00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 --2- 172.21.15.105:0/3334776405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a40d2530 0x7f26a40ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 >> 172.21.15.105:0/3334776405 conn(0x7f26a409cfb0 msgr2=0x7f26a40b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:09.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 shutdown_connections 2024-03-08T08:28:09.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.638+0000 7f26ac76c640 1 -- 172.21.15.105:0/3334776405 wait complete. 2024-03-08T08:28:09.649 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters' 2024-03-08T08:28:09.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.748+0000 7f9b4853e640 1 Processor -- start 2024-03-08T08:28:09.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.748+0000 7f9b4853e640 1 -- start start 2024-03-08T08:28:09.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.749+0000 7f9b4853e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b40072440 0x7f9b40071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.749+0000 7f9b4853e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b400b4d00 0x7f9b4015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.749+0000 7f9b4853e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9b4015b4a0 0x7f9b4006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.749+0000 7f9b4853e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4015b940 con 0x7f9b4015b4a0 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.749+0000 7f9b4853e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4015bc20 con 0x7f9b400b4d00 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.749+0000 7f9b4853e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b40070840 con 0x7f9b40072440 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b46adb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9b4015b4a0 0x7f9b4006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b46adb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9b4015b4a0 0x7f9b4006e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59408/0 (socket says 172.21.15.105:59408) 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b46adb640 1 -- 172.21.15.105:0/3507988105 learned_addr learned my addr 172.21.15.105:0/3507988105 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b45ad9640 1 --2- 172.21.15.105:0/3507988105 >> v2:172.21.15.144:3300/0 conn(0x7f9b400b4d00 0x7f9b4015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b462da640 1 --2- 172.21.15.105:0/3507988105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b40072440 0x7f9b40071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b46adb640 1 -- 172.21.15.105:0/3507988105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b40072440 msgr2=0x7f9b40071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b46adb640 1 --2- 172.21.15.105:0/3507988105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b40072440 0x7f9b40071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b46adb640 1 -- 172.21.15.105:0/3507988105 >> v2:172.21.15.144:3300/0 conn(0x7f9b400b4d00 msgr2=0x7f9b4015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b46adb640 1 --2- 172.21.15.105:0/3507988105 >> v2:172.21.15.144:3300/0 conn(0x7f9b400b4d00 0x7f9b4015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.750+0000 7f9b46adb640 1 -- 172.21.15.105:0/3507988105 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b40070b20 con 0x7f9b4015b4a0 2024-03-08T08:28:09.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b46adb640 1 --2- 172.21.15.105:0/3507988105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b4015b4a0 0x7f9b4006e0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b4006c7c0 tx=0x7f9b3002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c5df3c786fd984 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b452d8640 1 -- 172.21.15.105:0/3507988105 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3003b070 con 0x7f9b4015b4a0 2024-03-08T08:28:09.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b452d8640 1 -- 172.21.15.105:0/3507988105 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b30036440 con 0x7f9b4015b4a0 2024-03-08T08:28:09.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b452d8640 1 -- 172.21.15.105:0/3507988105 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30036810 con 0x7f9b4015b4a0 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 -- 172.21.15.105:0/3507988105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b4015b4a0 msgr2=0x7f9b4006e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3507988105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b4015b4a0 0x7f9b4006e0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b4006c7c0 tx=0x7f9b3002e0e0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 -- 172.21.15.105:0/3507988105 shutdown_connections 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3507988105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b4015b4a0 0x7f9b4006e0a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3507988105 >> v2:172.21.15.144:3300/0 conn(0x7f9b400b4d00 0x7f9b4015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3507988105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b40072440 0x7f9b40071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 -- 172.21.15.105:0/3507988105 >> 172.21.15.105:0/3507988105 conn(0x7f9b400b1740 msgr2=0x7f9b400b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 -- 172.21.15.105:0/3507988105 shutdown_connections 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.751+0000 7f9b4853e640 1 -- 172.21.15.105:0/3507988105 wait complete. 2024-03-08T08:28:09.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.752+0000 7f9b4853e640 1 Processor -- start 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.752+0000 7f9b4853e640 1 -- start start 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b4853e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b40072440 0x7f9b401456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b4853e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b400b4d00 0x7f9b4014bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b4853e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4015b4a0 0x7f9b4014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b4853e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b400729a0 con 0x7f9b40072440 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b4853e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b40072820 con 0x7f9b4015b4a0 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b4853e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4006e5e0 con 0x7f9b400b4d00 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b462da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b40072440 0x7f9b401456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b462da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b40072440 0x7f9b401456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59420/0 (socket says 172.21.15.105:59420) 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b462da640 1 -- 172.21.15.105:0/3916276613 learned_addr learned my addr 172.21.15.105:0/3916276613 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b46adb640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4015b4a0 0x7f9b4014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.753+0000 7f9b45ad9640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b400b4d00 0x7f9b4014bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b462da640 1 -- 172.21.15.105:0/3916276613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b400b4d00 msgr2=0x7f9b4014bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b462da640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b400b4d00 0x7f9b4014bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b462da640 1 -- 172.21.15.105:0/3916276613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4015b4a0 msgr2=0x7f9b4014c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b462da640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4015b4a0 0x7f9b4014c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b462da640 1 -- 172.21.15.105:0/3916276613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b401f70e0 con 0x7f9b40072440 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b462da640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b40072440 0x7f9b401456f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009db0 tx=0x7f9b3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b3effd640 1 -- 172.21.15.105:0/3916276613 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38018020 con 0x7f9b40072440 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b3effd640 1 -- 172.21.15.105:0/3916276613 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b38014480 con 0x7f9b40072440 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b401f73c0 con 0x7f9b40072440 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b3effd640 1 -- 172.21.15.105:0/3916276613 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38014850 con 0x7f9b40072440 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.754+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b401f7980 con 0x7f9b40072440 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.755+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b40075120 con 0x7f9b40072440 2024-03-08T08:28:09.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.755+0000 7f9b3effd640 1 -- 172.21.15.105:0/3916276613 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b38005d10 con 0x7f9b40072440 2024-03-08T08:28:09.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.755+0000 7f9b3effd640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9b2c06b4d0 0x7f9b2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:09.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.756+0000 7f9b3effd640 1 -- 172.21.15.105:0/3916276613 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b3808af00 con 0x7f9b40072440 2024-03-08T08:28:09.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.756+0000 7f9b45ad9640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9b2c06b4d0 0x7f9b2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:09.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.756+0000 7f9b45ad9640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9b2c06b4d0 0x7f9b2c06d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b40071f50 tx=0x7f9b34009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:09.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.760+0000 7f9b3effd640 1 -- 172.21.15.105:0/3916276613 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b38022030 con 0x7f9b40072440 2024-03-08T08:28:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:09.996+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]} v 0) v1 -- 0x7f9b4006ccb0 con 0x7f9b40072440 2024-03-08T08:28:10.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.782+0000 7f9b3effd640 1 -- 172.21.15.105:0/3916276613 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]}]=0 v53) v1 ==== 167+0+0 (secure 0 0 0) 0x7f9b38010040 con 0x7f9b40072440 2024-03-08T08:28:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9b2c06b4d0 msgr2=0x7f9b2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9b2c06b4d0 0x7f9b2c06d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9b40071f50 tx=0x7f9b34009190 comp rx=0 tx=0).stop 2024-03-08T08:28:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b40072440 msgr2=0x7f9b401456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b40072440 0x7f9b401456f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009db0 tx=0x7f9b3800c490 comp rx=0 tx=0).stop 2024-03-08T08:28:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 shutdown_connections 2024-03-08T08:28:10.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9b2c06b4d0 0x7f9b2c06d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b4015b4a0 0x7f9b4014c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b400b4d00 0x7f9b4014bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 --2- 172.21.15.105:0/3916276613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b40072440 0x7f9b401456f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 >> 172.21.15.105:0/3916276613 conn(0x7f9b400b1740 msgr2=0x7f9b400cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:10.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 shutdown_connections 2024-03-08T08:28:10.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.784+0000 7f9b4853e640 1 -- 172.21.15.105:0/3916276613 wait complete. 2024-03-08T08:28:10.798 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-08T08:28:10.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.903+0000 7f16d89a2640 1 Processor -- start 2024-03-08T08:28:10.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.903+0000 7f16d89a2640 1 -- start start 2024-03-08T08:28:10.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d89a2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d20c0 0x7f16d40d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:10.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d89a2640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16d415e120 0x7f16d40cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:10.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d89a2640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d40cf900 con 0x7f16d40d24a0 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d89a2640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d40cf6d0 con 0x7f16d415e120 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d89a2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40b06a0 con 0x7f16d40d20c0 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d37fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d20c0 0x7f16d40d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d2ffd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16d415e120 0x7f16d40cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d37fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d20c0 0x7f16d40d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:49212/0 (socket says 172.21.15.105:49212) 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d37fe640 1 -- 172.21.15.105:0/3527351109 learned_addr learned my addr 172.21.15.105:0/3527351109 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d3fff640 1 --1- 172.21.15.105:0/3527351109 >> v1:172.21.15.144:6789/0 conn(0x7f16d40d24a0 0x7f16d40d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:57656/0 (socket says 172.21.15.105:57656) 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d37fe640 1 -- 172.21.15.105:0/3527351109 >> v2:172.21.15.144:3300/0 conn(0x7f16d415e120 msgr2=0x7f16d40cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d37fe640 1 --2- 172.21.15.105:0/3527351109 >> v2:172.21.15.144:3300/0 conn(0x7f16d415e120 0x7f16d40cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.904+0000 7f16d37fe640 1 -- 172.21.15.105:0/3527351109 >> v1:172.21.15.144:6789/0 conn(0x7f16d40d24a0 legacy=0x7f16d40d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:10.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.905+0000 7f16d37fe640 1 -- 172.21.15.105:0/3527351109 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d40b08f0 con 0x7f16d40d20c0 2024-03-08T08:28:10.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.905+0000 7f16d37fe640 1 --2- 172.21.15.105:0/3527351109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d20c0 0x7f16d40d18c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009870 tx=0x7f16c402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd413c4d96e477e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.905+0000 7f16d27fc640 1 -- 172.21.15.105:0/3527351109 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4034070 con 0x7f16d40d20c0 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.905+0000 7f16d27fc640 1 -- 172.21.15.105:0/3527351109 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c4030840 con 0x7f16d40d20c0 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.905+0000 7f16d27fc640 1 -- 172.21.15.105:0/3527351109 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4030c10 con 0x7f16d40d20c0 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.906+0000 7f16d89a2640 1 -- 172.21.15.105:0/3527351109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d20c0 msgr2=0x7f16d40d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.906+0000 7f16d89a2640 1 --2- 172.21.15.105:0/3527351109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d20c0 0x7f16d40d18c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009870 tx=0x7f16c402b980 comp rx=0 tx=0).stop 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.906+0000 7f16d89a2640 1 -- 172.21.15.105:0/3527351109 shutdown_connections 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.906+0000 7f16d89a2640 1 --2- 172.21.15.105:0/3527351109 >> v2:172.21.15.144:3300/0 conn(0x7f16d415e120 0x7f16d40cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.906+0000 7f16d89a2640 1 --2- 172.21.15.105:0/3527351109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d20c0 0x7f16d40d18c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.906+0000 7f16d89a2640 1 -- 172.21.15.105:0/3527351109 >> 172.21.15.105:0/3527351109 conn(0x7f16d409cdb0 msgr2=0x7f16d40d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.906+0000 7f16d89a2640 1 -- 172.21.15.105:0/3527351109 shutdown_connections 2024-03-08T08:28:10.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.906+0000 7f16d89a2640 1 -- 172.21.15.105:0/3527351109 wait complete. 2024-03-08T08:28:10.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.907+0000 7f16d89a2640 1 Processor -- start 2024-03-08T08:28:10.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.907+0000 7f16d89a2640 1 -- start start 2024-03-08T08:28:10.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d89a2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16d40d20c0 0x7f16d4155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:10.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d89a2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d24a0 0x7f16d40b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d89a2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16d415e120 0x7f16d40b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d89a2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40d2920 con 0x7f16d415e120 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d89a2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40b0f40 con 0x7f16d40d20c0 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d89a2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40b23e0 con 0x7f16d40d24a0 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16d40d20c0 0x7f16d4155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16d415e120 0x7f16d40b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16d40d20c0 0x7f16d4155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:50828/0 (socket says 172.21.15.105:50828) 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d2ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d24a0 0x7f16d40b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.908+0000 7f16d37fe640 1 -- 172.21.15.105:0/1979368993 learned_addr learned my addr 172.21.15.105:0/1979368993 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d37fe640 1 -- 172.21.15.105:0/1979368993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d24a0 msgr2=0x7f16d40b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d37fe640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d24a0 0x7f16d40b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d37fe640 1 -- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16d415e120 msgr2=0x7f16d40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:10.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d37fe640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16d415e120 0x7f16d40b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d37fe640 1 -- 172.21.15.105:0/1979368993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4155860 con 0x7f16d40d20c0 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d2ffd640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d24a0 0x7f16d40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d3fff640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16d415e120 0x7f16d40b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d37fe640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16d40d20c0 0x7f16d4155320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16c4000c00 tx=0x7f16c4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16bbfff640 1 -- 172.21.15.105:0/1979368993 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4048070 con 0x7f16d40d20c0 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16bbfff640 1 -- 172.21.15.105:0/1979368993 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c4030640 con 0x7f16d40d20c0 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d415f310 con 0x7f16d40d20c0 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16bbfff640 1 -- 172.21.15.105:0/1979368993 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4034040 con 0x7f16d40d20c0 2024-03-08T08:28:10.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.909+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d415f8d0 con 0x7f16d40d20c0 2024-03-08T08:28:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.910+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d404efc0 con 0x7f16d40d20c0 2024-03-08T08:28:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.910+0000 7f16bbfff640 1 -- 172.21.15.105:0/1979368993 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16c404b070 con 0x7f16d40d20c0 2024-03-08T08:28:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.911+0000 7f16bbfff640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16c006b4d0 0x7f16c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.911+0000 7f16bbfff640 1 -- 172.21.15.105:0/1979368993 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16c40ac070 con 0x7f16d40d20c0 2024-03-08T08:28:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.911+0000 7f16d2ffd640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16c006b4d0 0x7f16c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:10.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.911+0000 7f16d2ffd640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16c006b4d0 0x7f16c006d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f16d40d0980 tx=0x7f16c8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:10.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:10.915+0000 7f16bbfff640 1 -- 172.21.15.105:0/1979368993 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16c407aa30 con 0x7f16d40d20c0 2024-03-08T08:28:11.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.085+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f16d409d2a0 con 0x7f16d40d20c0 2024-03-08T08:28:11.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.092+0000 7f16bbfff640 1 -- 172.21.15.105:0/1979368993 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f16c407ead0 con 0x7f16d40d20c0 2024-03-08T08:28:11.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16c006b4d0 msgr2=0x7f16c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16c006b4d0 0x7f16c006d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f16d40d0980 tx=0x7f16c8009190 comp rx=0 tx=0).stop 2024-03-08T08:28:11.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16d40d20c0 msgr2=0x7f16d4155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16d40d20c0 0x7f16d4155320 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16c4000c00 tx=0x7f16c4004730 comp rx=0 tx=0).stop 2024-03-08T08:28:11.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 shutdown_connections 2024-03-08T08:28:11.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f16c006b4d0 0x7f16c006d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f16d415e120 0x7f16d40b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16d40d24a0 0x7f16d40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 --2- 172.21.15.105:0/1979368993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16d40d20c0 0x7f16d4155320 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 >> 172.21.15.105:0/1979368993 conn(0x7f16d409cdb0 msgr2=0x7f16d406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:11.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 shutdown_connections 2024-03-08T08:28:11.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.094+0000 7f16d89a2640 1 -- 172.21.15.105:0/1979368993 wait complete. 2024-03-08T08:28:11.106 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-08T08:28:11.106 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T08:28:11.144 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T08:28:11.191 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-08T08:28:11.191 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:11.191 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:11.206 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:11.207 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:11.261 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:11.262 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:11.317 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:11.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.455+0000 7f9edaba2640 1 Processor -- start 2024-03-08T08:28:11.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.456+0000 7f9edaba2640 1 -- start start 2024-03-08T08:28:11.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.456+0000 7f9edaba2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 0x7f9ed40d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.456+0000 7f9edaba2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed40d1370 con 0x7f9ed41615e0 2024-03-08T08:28:11.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.456+0000 7f9edaba2640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed40b4650 con 0x7f9ed40ce630 2024-03-08T08:28:11.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.456+0000 7f9edaba2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40d1c60 con 0x7f9ed40d2710 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.456+0000 7f9ed9ba0640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9ed41615e0 0x7f9ed40d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41954/0 (socket says 172.21.15.105:41954) 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed9ba0640 1 -- 172.21.15.105:0/3293277300 learned_addr learned my addr 172.21.15.105:0/3293277300 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed939f640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 0x7f9ed40d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399350769 0 0) 0x7f9ec4001110 con 0x7f9ed41615e0 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec0003520 con 0x7f9ed41615e0 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904425757 0 0) 0x7f9ec0003520 con 0x7f9ed41615e0 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 msgr2=0x7f9ed40d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 0x7f9ed40d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 >> v1:172.21.15.144:6789/0 conn(0x7f9ed40ce630 legacy=0x7f9ed40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40d1980 con 0x7f9ed41615e0 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f9ec4001840 con 0x7f9ed41615e0 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 >> v1:172.21.15.105:6789/0 conn(0x7f9ed41615e0 legacy=0x7f9ed40d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ec0003a50 0x7f9ec0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 0x7f9ec0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0004d40 0x7f9ec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40d1c60 con 0x7f9ec0003a50 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec000bbf0 con 0x7f9ec0004d40 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec000bdf0 con 0x7f9ed40d2710 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed9ba0640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ec0003a50 0x7f9ec0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9eda3a1640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0004d40 0x7f9ec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed939f640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 0x7f9ec0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed9ba0640 1 -- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 msgr2=0x7f9ec0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed9ba0640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 0x7f9ec0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed9ba0640 1 -- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0004d40 msgr2=0x7f9ec00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed9ba0640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0004d40 0x7f9ec00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed9ba0640 1 -- 172.21.15.105:0/3293277300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec0003520 con 0x7f9ec0003a50 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9ed939f640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40d2710 0x7f9ec0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:11.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.457+0000 7f9eda3a1640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0004d40 0x7f9ec00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:11.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.458+0000 7f9ed9ba0640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ec0003a50 0x7f9ec0003e70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400c4c0 tx=0x7f9ec0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd0cd9a6d95539f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:11.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.458+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400c0e0 con 0x7f9ec0003a50 2024-03-08T08:28:11.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.458+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4030070 con 0x7f9ec0003a50 2024-03-08T08:28:11.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.458+0000 7f9ed8b9e640 1 -- 172.21.15.105:0/3293277300 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec40058d0 con 0x7f9ec0003a50 2024-03-08T08:28:11.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.459+0000 7f9edaba2640 1 -- 172.21.15.105:0/3293277300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ec0003a50 msgr2=0x7f9ec0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.459+0000 7f9edaba2640 1 --2- 172.21.15.105:0/3293277300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ec0003a50 0x7f9ec0003e70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400c4c0 tx=0x7f9ec0005830 comp rx=0 tx=0).stop 2024-03-08T08:28:11.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.459+0000 7f9eda3a1640 1 -- 172.21.15.105:0/3293277300 reap_dead start 2024-03-08T08:28:11.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.459+0000 7f9edaba2640 1 -- 172.21.15.105:0/3293277300 shutdown_connections 2024-03-08T08:28:11.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.459+0000 7f9edaba2640 1 -- 172.21.15.105:0/3293277300 >> 172.21.15.105:0/3293277300 conn(0x7f9ed40d3a20 msgr2=0x7f9ed40d3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:11.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.459+0000 7f9edaba2640 1 -- 172.21.15.105:0/3293277300 shutdown_connections 2024-03-08T08:28:11.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.459+0000 7f9edaba2640 1 -- 172.21.15.105:0/3293277300 wait complete. 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.460+0000 7f9edaba2640 1 Processor -- start 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.460+0000 7f9edaba2640 1 -- start start 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9edaba2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0003a50 0x7f9ed414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9edaba2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40b8890 0x7f9ed40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9edaba2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed40b8d10 0x7f9ed40b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9edaba2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b4fc0 con 0x7f9ed40b8d10 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9edaba2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b4da0 con 0x7f9ec0003a50 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9edaba2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b5140 con 0x7f9ed40b8890 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed40b8d10 0x7f9ed40b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed40b8d10 0x7f9ed40b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59448/0 (socket says 172.21.15.105:59448) 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 -- 172.21.15.105:0/117793445 learned_addr learned my addr 172.21.15.105:0/117793445 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9ed9ba0640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0003a50 0x7f9ed414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9ed939f640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40b8890 0x7f9ed40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 -- 172.21.15.105:0/117793445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40b8890 msgr2=0x7f9ed40b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40b8890 0x7f9ed40b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 -- 172.21.15.105:0/117793445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0003a50 msgr2=0x7f9ed414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0003a50 0x7f9ed414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 -- 172.21.15.105:0/117793445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40b7340 con 0x7f9ed40b8d10 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eda3a1640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed40b8d10 0x7f9ed40b1b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc000f80 tx=0x7f9ecc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.461+0000 7f9eca7fc640 1 -- 172.21.15.105:0/117793445 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc019070 con 0x7f9ed40b8d10 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.462+0000 7f9eca7fc640 1 -- 172.21.15.105:0/117793445 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc006b50 con 0x7f9ed40b8d10 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.462+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed40b7db0 con 0x7f9ed40b8d10 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.462+0000 7f9eca7fc640 1 -- 172.21.15.105:0/117793445 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc003970 con 0x7f9ed40b8d10 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.462+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed40b8370 con 0x7f9ed40b8d10 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.463+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed4072870 con 0x7f9ed40b8d10 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.463+0000 7f9eca7fc640 1 -- 172.21.15.105:0/117793445 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ecc01c070 con 0x7f9ed40b8d10 2024-03-08T08:28:11.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.463+0000 7f9eca7fc640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9eb806b4d0 0x7f9eb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.463+0000 7f9ed9ba0640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9eb806b4d0 0x7f9eb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.463+0000 7f9eca7fc640 1 -- 172.21.15.105:0/117793445 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ecc0896c0 con 0x7f9ed40b8d10 2024-03-08T08:28:11.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.464+0000 7f9ed9ba0640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9eb806b4d0 0x7f9eb806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4034000 tx=0x7f9ec4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:11.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.468+0000 7f9eca7fc640 1 -- 172.21.15.105:0/117793445 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ed4072870 con 0x7f9ed40b8d10 2024-03-08T08:28:11.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.637+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ed40b8570 con 0x7f9ed40b8d10 2024-03-08T08:28:11.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.638+0000 7f9eca7fc640 1 -- 172.21.15.105:0/117793445 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f9ed40b8570 con 0x7f9ed40b8d10 2024-03-08T08:28:11.639 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:11.640 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:11.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9eb806b4d0 msgr2=0x7f9eb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9eb806b4d0 0x7f9eb806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4034000 tx=0x7f9ec4009030 comp rx=0 tx=0).stop 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed40b8d10 msgr2=0x7f9ed40b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed40b8d10 0x7f9ed40b1b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc000f80 tx=0x7f9ecc00eb90 comp rx=0 tx=0).stop 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 shutdown_connections 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9eb806b4d0 0x7f9eb806d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ed40b8d10 0x7f9ed40b1b80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ed40b8890 0x7f9ed40b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 --2- 172.21.15.105:0/117793445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ec0003a50 0x7f9ed414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 >> 172.21.15.105:0/117793445 conn(0x7f9ed40d3a20 msgr2=0x7f9ed415eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 shutdown_connections 2024-03-08T08:28:11.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.640+0000 7f9edaba2640 1 -- 172.21.15.105:0/117793445 wait complete. 2024-03-08T08:28:11.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:11.653 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:11.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.755+0000 7f46be91e640 1 Processor -- start 2024-03-08T08:28:11.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.755+0000 7f46be91e640 1 -- start start 2024-03-08T08:28:11.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46be91e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46b8072440 0x7f46b8071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46be91e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b815b670 con 0x7f46b80b4d00 2024-03-08T08:28:11.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46be91e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b815b940 con 0x7f46b806e0a0 2024-03-08T08:28:11.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46be91e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b815bc20 con 0x7f46b8072440 2024-03-08T08:28:11.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46b7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46b8072440 0x7f46b8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46b7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f46b8072440 0x7f46b8071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59458/0 (socket says 172.21.15.105:59458) 2024-03-08T08:28:11.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46b77fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f46b80b4d00 0x7f46b815af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41966/0 (socket says 172.21.15.105:41966) 2024-03-08T08:28:11.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46b7fff640 1 -- 172.21.15.105:0/1433797484 learned_addr learned my addr 172.21.15.105:0/1433797484 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:11.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46b7fff640 1 -- 172.21.15.105:0/1433797484 >> v1:172.21.15.144:6789/0 conn(0x7f46b806e0a0 legacy=0x7f46b80cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46b7fff640 1 -- 172.21.15.105:0/1433797484 >> v1:172.21.15.105:6789/0 conn(0x7f46b80b4d00 legacy=0x7f46b815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.756+0000 7f46b7fff640 1 -- 172.21.15.105:0/1433797484 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8070840 con 0x7f46b8072440 2024-03-08T08:28:11.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.757+0000 7f46b7fff640 1 --2- 172.21.15.105:0/1433797484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b8072440 0x7f46b8071020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009870 tx=0x7f46a802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80e18d54acbecbc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:11.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.757+0000 7f46b6ffd640 1 -- 172.21.15.105:0/1433797484 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a802f840 con 0x7f46b8072440 2024-03-08T08:28:11.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.757+0000 7f46b6ffd640 1 -- 172.21.15.105:0/1433797484 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a802f9e0 con 0x7f46b8072440 2024-03-08T08:28:11.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.757+0000 7f46b6ffd640 1 -- 172.21.15.105:0/1433797484 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a802fdb0 con 0x7f46b8072440 2024-03-08T08:28:11.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.758+0000 7f46be91e640 1 -- 172.21.15.105:0/1433797484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b8072440 msgr2=0x7f46b8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.758+0000 7f46be91e640 1 --2- 172.21.15.105:0/1433797484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b8072440 0x7f46b8071020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009870 tx=0x7f46a802ba20 comp rx=0 tx=0).stop 2024-03-08T08:28:11.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.758+0000 7f46be91e640 1 -- 172.21.15.105:0/1433797484 shutdown_connections 2024-03-08T08:28:11.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.758+0000 7f46be91e640 1 --2- 172.21.15.105:0/1433797484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b8072440 0x7f46b8071020 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.758+0000 7f46be91e640 1 -- 172.21.15.105:0/1433797484 >> 172.21.15.105:0/1433797484 conn(0x7f46b80b1740 msgr2=0x7f46b80b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:11.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.758+0000 7f46be91e640 1 -- 172.21.15.105:0/1433797484 shutdown_connections 2024-03-08T08:28:11.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.758+0000 7f46be91e640 1 -- 172.21.15.105:0/1433797484 wait complete. 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.759+0000 7f46be91e640 1 Processor -- start 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.759+0000 7f46be91e640 1 -- start start 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46be91e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b806e0a0 0x7f46b815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46be91e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8072440 0x7f46b815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46be91e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46b80b4d00 0x7f46b81f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46be91e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b8072820 con 0x7f46b806e0a0 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46be91e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b8071560 con 0x7f46b8072440 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46be91e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b80729a0 con 0x7f46b80b4d00 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b806e0a0 0x7f46b815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b806e0a0 0x7f46b815ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59472/0 (socket says 172.21.15.105:59472) 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 -- 172.21.15.105:0/2275903384 learned_addr learned my addr 172.21.15.105:0/2275903384 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:11.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b77fe640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8072440 0x7f46b815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 -- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46b80b4d00 msgr2=0x7f46b81f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46bcebb640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46b80b4d00 0x7f46b81f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46b80b4d00 0x7f46b81f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 -- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8072440 msgr2=0x7f46b815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8072440 0x7f46b815cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 -- 172.21.15.105:0/2275903384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8142ec0 con 0x7f46b806e0a0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b77fe640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8072440 0x7f46b815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46bcebb640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46b80b4d00 0x7f46b81f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.760+0000 7f46b7fff640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b806e0a0 0x7f46b815ca30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46a8000c00 tx=0x7f46a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.761+0000 7f46b4ff9640 1 -- 172.21.15.105:0/2275903384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8041400 con 0x7f46b806e0a0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.761+0000 7f46b4ff9640 1 -- 172.21.15.105:0/2275903384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a8046070 con 0x7f46b806e0a0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.761+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b81431a0 con 0x7f46b806e0a0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.761+0000 7f46b4ff9640 1 -- 172.21.15.105:0/2275903384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8041800 con 0x7f46b806e0a0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.761+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b8143760 con 0x7f46b806e0a0 2024-03-08T08:28:11.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.762+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b8075120 con 0x7f46b806e0a0 2024-03-08T08:28:11.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.762+0000 7f46b4ff9640 1 -- 172.21.15.105:0/2275903384 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46a8040070 con 0x7f46b806e0a0 2024-03-08T08:28:11.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.762+0000 7f46b4ff9640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f46a406b4d0 0x7f46a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:11.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.762+0000 7f46b4ff9640 1 -- 172.21.15.105:0/2275903384 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46a80acad0 con 0x7f46b806e0a0 2024-03-08T08:28:11.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.762+0000 7f46b77fe640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f46a406b4d0 0x7f46a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:11.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.763+0000 7f46b77fe640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f46a406b4d0 0x7f46a406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f46b8155430 tx=0x7f46ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:11.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.767+0000 7f46b4ff9640 1 -- 172.21.15.105:0/2275903384 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46a8033030 con 0x7f46b806e0a0 2024-03-08T08:28:11.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.937+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46b80b1bc0 con 0x7f46b806e0a0 2024-03-08T08:28:11.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.937+0000 7f46b4ff9640 1 -- 172.21.15.105:0/2275903384 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46a807b950 con 0x7f46b806e0a0 2024-03-08T08:28:11.938 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:11.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f46a406b4d0 msgr2=0x7f46a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f46a406b4d0 0x7f46a406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f46b8155430 tx=0x7f46ac009040 comp rx=0 tx=0).stop 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b806e0a0 msgr2=0x7f46b815ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b806e0a0 0x7f46b815ca30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f46a8000c00 tx=0x7f46a8004730 comp rx=0 tx=0).stop 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 shutdown_connections 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f46a406b4d0 0x7f46a406d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46b80b4d00 0x7f46b81f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46b8072440 0x7f46b815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 --2- 172.21.15.105:0/2275903384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46b806e0a0 0x7f46b815ca30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 >> 172.21.15.105:0/2275903384 conn(0x7f46b80b1740 msgr2=0x7f46b80cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 shutdown_connections 2024-03-08T08:28:11.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:11.939+0000 7f46be91e640 1 -- 172.21.15.105:0/2275903384 wait complete. 2024-03-08T08:28:11.951 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:12.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.052+0000 7faa7fb5c640 1 Processor -- start 2024-03-08T08:28:12.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.053+0000 7faa7fb5c640 1 -- start start 2024-03-08T08:28:12.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.053+0000 7faa7fb5c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7806bcb0 0x7faa7806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.053+0000 7faa7fb5c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa78161850 0x7faa780d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.053+0000 7faa7fb5c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa780d48a0 con 0x7faa78161470 2024-03-08T08:28:12.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.053+0000 7faa7fb5c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa780d2ba0 con 0x7faa78161850 2024-03-08T08:28:12.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.053+0000 7faa7fb5c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa780d4b80 con 0x7faa7806bcb0 2024-03-08T08:28:12.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.054+0000 7faa7e0f9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa78161850 0x7faa780d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.054+0000 7faa7e0f9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa78161850 0x7faa780d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59478/0 (socket says 172.21.15.105:59478) 2024-03-08T08:28:12.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.054+0000 7faa7e0f9640 1 -- 172.21.15.105:0/914126991 learned_addr learned my addr 172.21.15.105:0/914126991 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:12.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.054+0000 7faa7d0f7640 1 --2- 172.21.15.105:0/914126991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7806bcb0 0x7faa7806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.054+0000 7faa7e0f9640 1 -- 172.21.15.105:0/914126991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7806bcb0 msgr2=0x7faa7806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:12.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.054+0000 7faa7e0f9640 1 --2- 172.21.15.105:0/914126991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7806bcb0 0x7faa7806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.054+0000 7faa7e0f9640 1 -- 172.21.15.105:0/914126991 >> v1:172.21.15.144:6789/0 conn(0x7faa78161470 legacy=0x7faa7806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.054+0000 7faa7e0f9640 1 -- 172.21.15.105:0/914126991 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7806c170 con 0x7faa78161850 2024-03-08T08:28:12.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7e0f9640 1 --2- 172.21.15.105:0/914126991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa780d2660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa74009870 tx=0x7faa7402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94984898e16af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7c8f6640 1 -- 172.21.15.105:0/914126991 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74035020 con 0x7faa78161850 2024-03-08T08:28:12.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7c8f6640 1 -- 172.21.15.105:0/914126991 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa74031640 con 0x7faa78161850 2024-03-08T08:28:12.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7c8f6640 1 -- 172.21.15.105:0/914126991 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7402d040 con 0x7faa78161850 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7fb5c640 1 -- 172.21.15.105:0/914126991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 msgr2=0x7faa780d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/914126991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa780d2660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa74009870 tx=0x7faa7402b8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7fb5c640 1 -- 172.21.15.105:0/914126991 shutdown_connections 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/914126991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa780d2660 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/914126991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7806bcb0 0x7faa7806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7fb5c640 1 -- 172.21.15.105:0/914126991 >> 172.21.15.105:0/914126991 conn(0x7faa780d3ce0 msgr2=0x7faa780d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7fb5c640 1 -- 172.21.15.105:0/914126991 shutdown_connections 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.055+0000 7faa7fb5c640 1 -- 172.21.15.105:0/914126991 wait complete. 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.056+0000 7faa7fb5c640 1 Processor -- start 2024-03-08T08:28:12.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7fb5c640 1 -- start start 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7fb5c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7806bcb0 0x7faa780a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7fb5c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78161470 0x7faa780ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7fb5c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa7809f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7fb5c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7806cf40 con 0x7faa78161850 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7fb5c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7806d370 con 0x7faa7806bcb0 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7fb5c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7809db40 con 0x7faa78161470 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7e0f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa7809f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7e0f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa7809f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59490/0 (socket says 172.21.15.105:59490) 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.057+0000 7faa7e0f9640 1 -- 172.21.15.105:0/3874647607 learned_addr learned my addr 172.21.15.105:0/3874647607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7e0f9640 1 -- 172.21.15.105:0/3874647607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78161470 msgr2=0x7faa780ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7d0f7640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78161470 0x7faa780ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7d8f8640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7806bcb0 0x7faa780a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7e0f9640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78161470 0x7faa780ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7e0f9640 1 -- 172.21.15.105:0/3874647607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7806bcb0 msgr2=0x7faa780a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7e0f9640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7806bcb0 0x7faa780a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7e0f9640 1 -- 172.21.15.105:0/3874647607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7809dd90 con 0x7faa78161850 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7e0f9640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa7809f760 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa74009870 tx=0x7faa740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa6e7fc640 1 -- 172.21.15.105:0/3874647607 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74040070 con 0x7faa78161850 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa6e7fc640 1 -- 172.21.15.105:0/3874647607 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa740399f0 con 0x7faa78161850 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7809e070 con 0x7faa78161850 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa6e7fc640 1 -- 172.21.15.105:0/3874647607 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74035040 con 0x7faa78161850 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.058+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa78155110 con 0x7faa78161850 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.059+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa78075120 con 0x7faa78161850 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.059+0000 7faa6e7fc640 1 -- 172.21.15.105:0/3874647607 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa7404b060 con 0x7faa78161850 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.060+0000 7faa6e7fc640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7faa5c06b500 0x7faa5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.060+0000 7faa7d8f8640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7faa5c06b500 0x7faa5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.060+0000 7faa6e7fc640 1 -- 172.21.15.105:0/3874647607 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa740ab8f0 con 0x7faa78161850 2024-03-08T08:28:12.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.060+0000 7faa7d8f8640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7faa5c06b500 0x7faa5c06d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa70007fd0 tx=0x7faa70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.064+0000 7faa6e7fc640 1 -- 172.21.15.105:0/3874647607 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa7407a800 con 0x7faa78161850 2024-03-08T08:28:12.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.236+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa780b5540 con 0x7faa78161850 2024-03-08T08:28:12.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.236+0000 7faa6e7fc640 1 -- 172.21.15.105:0/3874647607 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa7407a100 con 0x7faa78161850 2024-03-08T08:28:12.237 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:12.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7faa5c06b500 msgr2=0x7faa5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7faa5c06b500 0x7faa5c06d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa70007fd0 tx=0x7faa70009040 comp rx=0 tx=0).stop 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 msgr2=0x7faa7809f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa7809f760 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa74009870 tx=0x7faa740046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 shutdown_connections 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7faa5c06b500 0x7faa5c06d9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa78161850 0x7faa7809f760 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78161470 0x7faa780ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 --2- 172.21.15.105:0/3874647607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7806bcb0 0x7faa780a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 >> 172.21.15.105:0/3874647607 conn(0x7faa780d3ce0 msgr2=0x7faa7806b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 shutdown_connections 2024-03-08T08:28:12.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.238+0000 7faa7fb5c640 1 -- 172.21.15.105:0/3874647607 wait complete. 2024-03-08T08:28:12.250 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.351+0000 7fdf26b1e640 1 Processor -- start 2024-03-08T08:28:12.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.351+0000 7fdf26b1e640 1 -- start start 2024-03-08T08:28:12.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf26b1e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf200ce4c0 0x7fdf200ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf26b1e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf200ce180 con 0x7fdf200cdc50 2024-03-08T08:28:12.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf26b1e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf200b15b0 con 0x7fdf200b2090 2024-03-08T08:28:12.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf26b1e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf200cd840 con 0x7fdf200ce4c0 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdf200cdc50 0x7fdf200cede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41976/0 (socket says 172.21.15.105:41976) 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1ffff640 1 -- 172.21.15.105:0/2080618600 learned_addr learned my addr 172.21.15.105:0/2080618600 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600007718 0 0) 0x7fdf200ce180 con 0x7fdf200cdc50 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf10003520 con 0x7fdf200cdc50 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf248ba640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf200ce4c0 0x7fdf200ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543820072 0 0) 0x7fdf10003520 con 0x7fdf200cdc50 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf200ce4c0 msgr2=0x7fdf200ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf200ce4c0 0x7fdf200ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 >> v1:172.21.15.144:6789/0 conn(0x7fdf200b2090 legacy=0x7fdf200b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf200b1890 con 0x7fdf200cdc50 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fdf0c001670 con 0x7fdf200cdc50 2024-03-08T08:28:12.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 >> v1:172.21.15.105:6789/0 conn(0x7fdf200cdc50 legacy=0x7fdf200cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf100039e0 0x7fdf10003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf10004800 0x7fdf10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf100051c0 0x7fdf1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1000c090 con 0x7fdf100051c0 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1000c210 con 0x7fdf10004800 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1000c410 con 0x7fdf100039e0 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf248ba640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf200ce4c0 0x7fdf200ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf250bb640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf100051c0 0x7fdf1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.352+0000 7fdf1ffff640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf10004800 0x7fdf10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.353+0000 7fdf248ba640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf100039e0 0x7fdf10003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.353+0000 7fdf250bb640 1 -- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf100039e0 msgr2=0x7fdf10003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.353+0000 7fdf250bb640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf100039e0 0x7fdf10003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.353+0000 7fdf250bb640 1 -- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf10004800 msgr2=0x7fdf10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.353+0000 7fdf250bb640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf10004800 0x7fdf10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.353+0000 7fdf250bb640 1 -- 172.21.15.105:0/2080618600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10003520 con 0x7fdf100051c0 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.353+0000 7fdf250bb640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf100051c0 0x7fdf1000b9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf18001e50 tx=0x7fdf10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad25dde06da7895 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.353+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18001a70 con 0x7fdf100051c0 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18003b70 con 0x7fdf100051c0 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf1f7fe640 1 -- 172.21.15.105:0/2080618600 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf180050d0 con 0x7fdf100051c0 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2080618600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf100051c0 msgr2=0x7fdf1000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf26b1e640 1 --2- 172.21.15.105:0/2080618600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf100051c0 0x7fdf1000b9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf18001e50 tx=0x7fdf10005830 comp rx=0 tx=0).stop 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf250bb640 1 -- 172.21.15.105:0/2080618600 reap_dead start 2024-03-08T08:28:12.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2080618600 shutdown_connections 2024-03-08T08:28:12.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2080618600 >> 172.21.15.105:0/2080618600 conn(0x7fdf200b37a0 msgr2=0x7fdf200b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:12.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2080618600 shutdown_connections 2024-03-08T08:28:12.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.354+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2080618600 wait complete. 2024-03-08T08:28:12.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.355+0000 7fdf26b1e640 1 Processor -- start 2024-03-08T08:28:12.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.355+0000 7fdf26b1e640 1 -- start start 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf26b1e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2014c460 0x7fdf2009e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf26b1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2009f9d0 0x7fdf2009bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf26b1e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2014c840 0x7fdf2009c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf26b1e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf200b12a0 con 0x7fdf2009f9d0 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf26b1e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2015c090 con 0x7fdf2014c460 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf26b1e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2015c210 con 0x7fdf2014c840 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf1ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2009f9d0 0x7fdf2009bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf1ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2009f9d0 0x7fdf2009bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59504/0 (socket says 172.21.15.105:59504) 2024-03-08T08:28:12.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf1ffff640 1 -- 172.21.15.105:0/2805301739 learned_addr learned my addr 172.21.15.105:0/2805301739 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf248ba640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2014c460 0x7fdf2009e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf250bb640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2014c840 0x7fdf2009c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf1ffff640 1 -- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2014c840 msgr2=0x7fdf2009c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf1ffff640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2014c840 0x7fdf2009c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf1ffff640 1 -- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2014c460 msgr2=0x7fdf2009e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf1ffff640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2014c460 0x7fdf2009e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf1ffff640 1 -- 172.21.15.105:0/2805301739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2009e930 con 0x7fdf2009f9d0 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.356+0000 7fdf248ba640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2014c460 0x7fdf2009e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.357+0000 7fdf1ffff640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2009f9d0 0x7fdf2009bb90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c007290 tx=0x7fdf0c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.357+0000 7fdf1d7fa640 1 -- 172.21.15.105:0/2805301739 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00f8e0 con 0x7fdf2009f9d0 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.357+0000 7fdf1d7fa640 1 -- 172.21.15.105:0/2805301739 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c018070 con 0x7fdf2009f9d0 2024-03-08T08:28:12.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.357+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2009ec10 con 0x7fdf2009f9d0 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.357+0000 7fdf1d7fa640 1 -- 172.21.15.105:0/2805301739 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00fc80 con 0x7fdf2009f9d0 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.357+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2009f1d0 con 0x7fdf2009f9d0 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.358+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf20075120 con 0x7fdf2009f9d0 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.358+0000 7fdf1d7fa640 1 -- 172.21.15.105:0/2805301739 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf0c01d460 con 0x7fdf2009f9d0 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.359+0000 7fdf1d7fa640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf0406b500 0x7fdf0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.359+0000 7fdf1d7fa640 1 -- 172.21.15.105:0/2805301739 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf0c08a310 con 0x7fdf2009f9d0 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.359+0000 7fdf248ba640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf0406b500 0x7fdf0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.359+0000 7fdf248ba640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf0406b500 0x7fdf0406d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdf14006fd0 tx=0x7fdf14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.363+0000 7fdf1d7fa640 1 -- 172.21.15.105:0/2805301739 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf0c0591a0 con 0x7fdf2009f9d0 2024-03-08T08:28:12.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.531+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf200b2910 con 0x7fdf2009f9d0 2024-03-08T08:28:12.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.532+0000 7fdf1d7fa640 1 -- 172.21.15.105:0/2805301739 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf0c058aa0 con 0x7fdf2009f9d0 2024-03-08T08:28:12.533 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf0406b500 msgr2=0x7fdf0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf0406b500 0x7fdf0406d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdf14006fd0 tx=0x7fdf14008040 comp rx=0 tx=0).stop 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2009f9d0 msgr2=0x7fdf2009bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2009f9d0 0x7fdf2009bb90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c007290 tx=0x7fdf0c007750 comp rx=0 tx=0).stop 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 shutdown_connections 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdf0406b500 0x7fdf0406d9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2014c840 0x7fdf2009c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf2009f9d0 0x7fdf2009bb90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 --2- 172.21.15.105:0/2805301739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2014c460 0x7fdf2009e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 >> 172.21.15.105:0/2805301739 conn(0x7fdf200b37a0 msgr2=0x7fdf20159230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 shutdown_connections 2024-03-08T08:28:12.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.534+0000 7fdf26b1e640 1 -- 172.21.15.105:0/2805301739 wait complete. 2024-03-08T08:28:12.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T08:28:12.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.645+0000 7f84b5207640 1 Processor -- start 2024-03-08T08:28:12.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.645+0000 7f84b5207640 1 -- start start 2024-03-08T08:28:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.646+0000 7f84b5207640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f84b00d06e0 0x7f84b015f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.646+0000 7f84b5207640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00b2740 0x7f84b00b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.646+0000 7f84b5207640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b00b2c20 con 0x7f84b00cfdb0 2024-03-08T08:28:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.646+0000 7f84b5207640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b00af4c0 con 0x7f84b00d06e0 2024-03-08T08:28:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.646+0000 7f84b5207640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b00af7a0 con 0x7f84b00b2740 2024-03-08T08:28:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.646+0000 7f84af59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00b2740 0x7f84b00b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.646+0000 7f84ae59c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f84b00d06e0 0x7f84b015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.647+0000 7f84af59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00b2740 0x7f84b00b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:49328/0 (socket says 172.21.15.105:49328) 2024-03-08T08:28:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.647+0000 7f84aed9d640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f84b00cfdb0 0x7f84b015ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:57708/0 (socket says 172.21.15.105:57708) 2024-03-08T08:28:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.647+0000 7f84af59e640 1 -- 172.21.15.105:0/3871127052 learned_addr learned my addr 172.21.15.105:0/3871127052 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.647+0000 7f84af59e640 1 -- 172.21.15.105:0/3871127052 >> v2:172.21.15.144:3300/0 conn(0x7f84b00d06e0 msgr2=0x7f84b015f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.647+0000 7f84af59e640 1 --2- 172.21.15.105:0/3871127052 >> v2:172.21.15.144:3300/0 conn(0x7f84b00d06e0 0x7f84b015f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.647+0000 7f84af59e640 1 -- 172.21.15.105:0/3871127052 >> v1:172.21.15.144:6789/0 conn(0x7f84b00cfdb0 legacy=0x7f84b015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.647+0000 7f84af59e640 1 -- 172.21.15.105:0/3871127052 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b00cc490 con 0x7f84b00b2740 2024-03-08T08:28:12.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.647+0000 7f84ae59c640 1 --2- 172.21.15.105:0/3871127052 >> v2:172.21.15.144:3300/0 conn(0x7f84b00d06e0 0x7f84b015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:12.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84af59e640 1 --2- 172.21.15.105:0/3871127052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00b2740 0x7f84b00b1f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009e50 tx=0x7f84a802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6086400a99cd753 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84add9b640 1 -- 172.21.15.105:0/3871127052 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a802ca70 con 0x7f84b00b2740 2024-03-08T08:28:12.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84add9b640 1 -- 172.21.15.105:0/3871127052 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a802cc10 con 0x7f84b00b2740 2024-03-08T08:28:12.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84add9b640 1 -- 172.21.15.105:0/3871127052 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a80343f0 con 0x7f84b00b2740 2024-03-08T08:28:12.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84b5207640 1 -- 172.21.15.105:0/3871127052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00b2740 msgr2=0x7f84b00b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84b5207640 1 --2- 172.21.15.105:0/3871127052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00b2740 0x7f84b00b1f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009e50 tx=0x7f84a802bed0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84b5207640 1 -- 172.21.15.105:0/3871127052 shutdown_connections 2024-03-08T08:28:12.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84b5207640 1 --2- 172.21.15.105:0/3871127052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00b2740 0x7f84b00b1f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84b5207640 1 --2- 172.21.15.105:0/3871127052 >> v2:172.21.15.144:3300/0 conn(0x7f84b00d06e0 0x7f84b015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84b5207640 1 -- 172.21.15.105:0/3871127052 >> 172.21.15.105:0/3871127052 conn(0x7f84b00b36a0 msgr2=0x7f84b00b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:12.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84b5207640 1 -- 172.21.15.105:0/3871127052 shutdown_connections 2024-03-08T08:28:12.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.648+0000 7f84b5207640 1 -- 172.21.15.105:0/3871127052 wait complete. 2024-03-08T08:28:12.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.650+0000 7f84b5207640 1 Processor -- start 2024-03-08T08:28:12.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.650+0000 7f84b5207640 1 -- start start 2024-03-08T08:28:12.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.650+0000 7f84b5207640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00b2740 0x7f84b00b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.650+0000 7f84b5207640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b00cfdb0 0x7f84b015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84b5207640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00d06e0 0x7f84b0154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84b5207640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0165b70 con 0x7f84b00cfdb0 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84b5207640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b00d03a0 con 0x7f84b00b2740 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84b5207640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0165cf0 con 0x7f84b00d06e0 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b00cfdb0 0x7f84b015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b00cfdb0 0x7f84b015d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59520/0 (socket says 172.21.15.105:59520) 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 -- 172.21.15.105:0/777277114 learned_addr learned my addr 172.21.15.105:0/777277114 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84af59e640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00d06e0 0x7f84b0154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84aed9d640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00b2740 0x7f84b00b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 -- 172.21.15.105:0/777277114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00d06e0 msgr2=0x7f84b0154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00d06e0 0x7f84b0154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 -- 172.21.15.105:0/777277114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00b2740 msgr2=0x7f84b00b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00b2740 0x7f84b00b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 -- 172.21.15.105:0/777277114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b0157330 con 0x7f84b00cfdb0 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84ae59c640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b00cfdb0 0x7f84b015d2e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e9b0 tx=0x7f84a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84977fe640 1 -- 172.21.15.105:0/777277114 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4017070 con 0x7f84b00cfdb0 2024-03-08T08:28:12.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84977fe640 1 -- 172.21.15.105:0/777277114 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a400ca00 con 0x7f84b00cfdb0 2024-03-08T08:28:12.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84977fe640 1 -- 172.21.15.105:0/777277114 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400cdf0 con 0x7f84b00cfdb0 2024-03-08T08:28:12.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b01566c0 con 0x7f84b00cfdb0 2024-03-08T08:28:12.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.651+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b0156c80 con 0x7f84b00cfdb0 2024-03-08T08:28:12.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.652+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b0075120 con 0x7f84b00cfdb0 2024-03-08T08:28:12.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.653+0000 7f84977fe640 1 -- 172.21.15.105:0/777277114 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84a401a070 con 0x7f84b00cfdb0 2024-03-08T08:28:12.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.653+0000 7f84977fe640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f849c06b4d0 0x7f849c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:12.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.653+0000 7f84977fe640 1 -- 172.21.15.105:0/777277114 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a4089d10 con 0x7f84b00cfdb0 2024-03-08T08:28:12.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.653+0000 7f84aed9d640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f849c06b4d0 0x7f849c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:12.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.654+0000 7f84aed9d640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f849c06b4d0 0x7f849c06d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f84b00c87c0 tx=0x7f84a0007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:12.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.657+0000 7f84977fe640 1 -- 172.21.15.105:0/777277114 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84b0157330 con 0x7f84b00cfdb0 2024-03-08T08:28:12.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.843+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f84b00b79d0 con 0x7f84b00cfdb0 2024-03-08T08:28:12.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.843+0000 7f84977fe640 1 -- 172.21.15.105:0/777277114 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f84a4058540 con 0x7f84b00cfdb0 2024-03-08T08:28:12.844 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-08T08:28:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f849c06b4d0 msgr2=0x7f849c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f849c06b4d0 0x7f849c06d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f84b00c87c0 tx=0x7f84a0007e80 comp rx=0 tx=0).stop 2024-03-08T08:28:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b00cfdb0 msgr2=0x7f84b015d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b00cfdb0 0x7f84b015d2e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e9b0 tx=0x7f84a400ee70 comp rx=0 tx=0).stop 2024-03-08T08:28:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 shutdown_connections 2024-03-08T08:28:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f849c06b4d0 0x7f849c06d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84b00d06e0 0x7f84b0154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b00cfdb0 0x7f84b015d2e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 --2- 172.21.15.105:0/777277114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00b2740 0x7f84b00b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 >> 172.21.15.105:0/777277114 conn(0x7f84b00b36a0 msgr2=0x7f84b00cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 shutdown_connections 2024-03-08T08:28:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:12.845+0000 7f84b5207640 1 -- 172.21.15.105:0/777277114 wait complete. 2024-03-08T08:28:12.857 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-08T08:28:12.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:12.857 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-08T08:28:12.885 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:28:12.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:12.885 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-08T08:28:12.956 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:12.956 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:12.975 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:12.975 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:13.033 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:13.033 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:13.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:13.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.190+0000 7f9f73f27640 1 Processor -- start 2024-03-08T08:28:13.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.191+0000 7f9f73f27640 1 -- start start 2024-03-08T08:28:13.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.191+0000 7f9f73f27640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 0x7f9f6c06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.191+0000 7f9f73f27640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9f6c0b5ec0 0x7f9f6c06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f73f27640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6c06c4d0 con 0x7f9f6c06bfa0 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f73f27640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f6c15de20 con 0x7f9f6c0b5ec0 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f73f27640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c15e100 con 0x7f9f6c15a200 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f714c2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9f6c06bfa0 0x7f9f6c0722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41990/0 (socket says 172.21.15.105:41990) 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f714c2640 1 -- 172.21.15.105:0/2937183454 learned_addr learned my addr 172.21.15.105:0/2937183454 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450735579 0 0) 0x7f9f6c06c4d0 con 0x7f9f6c06bfa0 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f58003520 con 0x7f9f6c06bfa0 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f71cc3640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 0x7f9f6c06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f724c4640 1 --2- 172.21.15.105:0/2937183454 >> v2:172.21.15.144:3300/0 conn(0x7f9f6c0b5ec0 0x7f9f6c06d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 227300144 0 0) 0x7f9f58003520 con 0x7f9f6c06bfa0 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 msgr2=0x7f9f6c06cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 0x7f9f6c06cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 >> v2:172.21.15.144:3300/0 conn(0x7f9f6c0b5ec0 msgr2=0x7f9f6c06d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 --2- 172.21.15.105:0/2937183454 >> v2:172.21.15.144:3300/0 conn(0x7f9f6c0b5ec0 0x7f9f6c06d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c15e3e0 con 0x7f9f6c06bfa0 2024-03-08T08:28:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f9f5c001670 con 0x7f9f6c06bfa0 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 >> v1:172.21.15.105:6789/0 conn(0x7f9f6c06bfa0 legacy=0x7f9f6c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f580039b0 0x7f9f58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 0x7f9f58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f58004d40 0x7f9f580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c15de20 con 0x7f9f580039b0 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c15e100 con 0x7f9f58004d40 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5800bb20 con 0x7f9f6c15a200 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f71cc3640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f580039b0 0x7f9f58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.192+0000 7f9f714c2640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 0x7f9f58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.193+0000 7f9f724c4640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f58004d40 0x7f9f580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.193+0000 7f9f71cc3640 1 -- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 msgr2=0x7f9f58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.193+0000 7f9f71cc3640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 0x7f9f58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.193+0000 7f9f71cc3640 1 -- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f58004d40 msgr2=0x7f9f580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.193+0000 7f9f71cc3640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f58004d40 0x7f9f580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.193+0000 7f9f71cc3640 1 -- 172.21.15.105:0/2937183454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f58003520 con 0x7f9f580039b0 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.193+0000 7f9f714c2640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f6c15a200 0x7f9f58004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.193+0000 7f9f71cc3640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f580039b0 0x7f9f58003dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400ace0 tx=0x7f9f58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac49803f6a1c01b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64033070 con 0x7f9f580039b0 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6402db60 con 0x7f9f580039b0 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f70cc1640 1 -- 172.21.15.105:0/2937183454 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400a8e0 con 0x7f9f580039b0 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f73f27640 1 -- 172.21.15.105:0/2937183454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f580039b0 msgr2=0x7f9f58003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f73f27640 1 --2- 172.21.15.105:0/2937183454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f580039b0 0x7f9f58003dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400ace0 tx=0x7f9f58005830 comp rx=0 tx=0).stop 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f724c4640 1 -- 172.21.15.105:0/2937183454 reap_dead start 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f73f27640 1 -- 172.21.15.105:0/2937183454 shutdown_connections 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f73f27640 1 -- 172.21.15.105:0/2937183454 >> 172.21.15.105:0/2937183454 conn(0x7f9f6c0b50f0 msgr2=0x7f9f6c0b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f73f27640 1 -- 172.21.15.105:0/2937183454 shutdown_connections 2024-03-08T08:28:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.194+0000 7f9f73f27640 1 -- 172.21.15.105:0/2937183454 wait complete. 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.195+0000 7f9f73f27640 1 Processor -- start 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.195+0000 7f9f73f27640 1 -- start start 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f73f27640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f580039b0 0x7f9f6c15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f73f27640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c0cee60 0x7f9f6c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f73f27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6c1563b0 0x7f9f6c156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f73f27640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c15a5e0 con 0x7f9f6c1563b0 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f73f27640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c15ebe0 con 0x7f9f6c0cee60 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f73f27640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c15a760 con 0x7f9f580039b0 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f724c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6c1563b0 0x7f9f6c156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f724c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6c1563b0 0x7f9f6c156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59528/0 (socket says 172.21.15.105:59528) 2024-03-08T08:28:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f724c4640 1 -- 172.21.15.105:0/1502967120 learned_addr learned my addr 172.21.15.105:0/1502967120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f71cc3640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f580039b0 0x7f9f6c15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f714c2640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c0cee60 0x7f9f6c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f724c4640 1 -- 172.21.15.105:0/1502967120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f580039b0 msgr2=0x7f9f6c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f724c4640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f580039b0 0x7f9f6c15c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f724c4640 1 -- 172.21.15.105:0/1502967120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c0cee60 msgr2=0x7f9f6c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f724c4640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c0cee60 0x7f9f6c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.196+0000 7f9f724c4640 1 -- 172.21.15.105:0/1502967120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c15d7c0 con 0x7f9f6c1563b0 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.197+0000 7f9f724c4640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6c1563b0 0x7f9f6c156830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c073360 tx=0x7f9f6800f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.197+0000 7f9f627fc640 1 -- 172.21.15.105:0/1502967120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68017020 con 0x7f9f6c1563b0 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.197+0000 7f9f627fc640 1 -- 172.21.15.105:0/1502967120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f68004240 con 0x7f9f6c1563b0 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.197+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6c1f26a0 con 0x7f9f6c1563b0 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.197+0000 7f9f627fc640 1 -- 172.21.15.105:0/1502967120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68003610 con 0x7f9f6c1563b0 2024-03-08T08:28:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.197+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6c15b9d0 con 0x7f9f6c1563b0 2024-03-08T08:28:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.198+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6c075120 con 0x7f9f6c1563b0 2024-03-08T08:28:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.198+0000 7f9f627fc640 1 -- 172.21.15.105:0/1502967120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f6801a020 con 0x7f9f6c1563b0 2024-03-08T08:28:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.199+0000 7f9f627fc640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f5006b500 0x7f9f5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.199+0000 7f9f71cc3640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f5006b500 0x7f9f5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.199+0000 7f9f627fc640 1 -- 172.21.15.105:0/1502967120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f68089d50 con 0x7f9f6c1563b0 2024-03-08T08:28:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.199+0000 7f9f71cc3640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f5006b500 0x7f9f5006d9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f64038000 tx=0x7f9f64033000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.203+0000 7f9f627fc640 1 -- 172.21.15.105:0/1502967120 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f6808d030 con 0x7f9f6c1563b0 2024-03-08T08:28:13.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.375+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f6c0b6330 con 0x7f9f6c1563b0 2024-03-08T08:28:13.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.376+0000 7f9f627fc640 1 -- 172.21.15.105:0/1502967120 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f9f68021020 con 0x7f9f6c1563b0 2024-03-08T08:28:13.377 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:13.377 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:13.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f5006b500 msgr2=0x7f9f5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f5006b500 0x7f9f5006d9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f64038000 tx=0x7f9f64033000 comp rx=0 tx=0).stop 2024-03-08T08:28:13.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6c1563b0 msgr2=0x7f9f6c156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6c1563b0 0x7f9f6c156830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c073360 tx=0x7f9f6800f7e0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 shutdown_connections 2024-03-08T08:28:13.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f9f5006b500 0x7f9f5006d9c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f6c1563b0 0x7f9f6c156830 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c0cee60 0x7f9f6c0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 --2- 172.21.15.105:0/1502967120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f580039b0 0x7f9f6c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 >> 172.21.15.105:0/1502967120 conn(0x7f9f6c0b50f0 msgr2=0x7f9f6c155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:13.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 shutdown_connections 2024-03-08T08:28:13.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.378+0000 7f9f73f27640 1 -- 172.21.15.105:0/1502967120 wait complete. 2024-03-08T08:28:13.390 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:13.390 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.492+0000 7f250bc6b640 1 Processor -- start 2024-03-08T08:28:13.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.492+0000 7f250bc6b640 1 -- start start 2024-03-08T08:28:13.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f250bc6b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2504147510 0x7f2504071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f250bc6b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25040b4660 0x7f250415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f250bc6b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f250406d920 0x7f250406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f250bc6b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25040ad870 con 0x7f25040b4660 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f250bc6b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25040b4af0 con 0x7f2504147510 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f250bc6b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25040b4dc0 con 0x7f250406d920 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f2509206640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25040b4660 0x7f250415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f2509206640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25040b4660 0x7f250415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42142/0 (socket says 172.21.15.105:42142) 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f2509206640 1 -- 172.21.15.105:0/750674804 learned_addr learned my addr 172.21.15.105:0/750674804 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f2509206640 1 -- 172.21.15.105:0/750674804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f250406d920 msgr2=0x7f250406ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f2509206640 1 --2- 172.21.15.105:0/750674804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f250406d920 0x7f250406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f2509206640 1 -- 172.21.15.105:0/750674804 >> v2:172.21.15.144:3300/0 conn(0x7f2504147510 msgr2=0x7f2504071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.493+0000 7f2509206640 1 --2- 172.21.15.105:0/750674804 >> v2:172.21.15.144:3300/0 conn(0x7f2504147510 0x7f2504071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.494+0000 7f2509206640 1 -- 172.21.15.105:0/750674804 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040adb10 con 0x7f25040b4660 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.494+0000 7f2509206640 1 --2- 172.21.15.105:0/750674804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25040b4660 0x7f250415e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25040af560 tx=0x7f24f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8907cc0899abac0f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.494+0000 7f2508a05640 1 -- 172.21.15.105:0/750674804 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f402e070 con 0x7f25040b4660 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.494+0000 7f2508a05640 1 -- 172.21.15.105:0/750674804 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f4037070 con 0x7f25040b4660 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.494+0000 7f2508a05640 1 -- 172.21.15.105:0/750674804 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4031a00 con 0x7f25040b4660 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 -- 172.21.15.105:0/750674804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25040b4660 msgr2=0x7f250415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 --2- 172.21.15.105:0/750674804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25040b4660 0x7f250415e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25040af560 tx=0x7f24f402b880 comp rx=0 tx=0).stop 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 -- 172.21.15.105:0/750674804 shutdown_connections 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 --2- 172.21.15.105:0/750674804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f250406d920 0x7f250406ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 --2- 172.21.15.105:0/750674804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25040b4660 0x7f250415e2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 --2- 172.21.15.105:0/750674804 >> v2:172.21.15.144:3300/0 conn(0x7f2504147510 0x7f2504071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 -- 172.21.15.105:0/750674804 >> 172.21.15.105:0/750674804 conn(0x7f25040b1740 msgr2=0x7f25040b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 -- 172.21.15.105:0/750674804 shutdown_connections 2024-03-08T08:28:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.495+0000 7f250bc6b640 1 -- 172.21.15.105:0/750674804 wait complete. 2024-03-08T08:28:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.496+0000 7f250bc6b640 1 Processor -- start 2024-03-08T08:28:13.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.496+0000 7f250bc6b640 1 -- start start 2024-03-08T08:28:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f250bc6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f250406d920 0x7f25041f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f250bc6b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25040b4660 0x7f25040b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f250bc6b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2504147510 0x7f25040b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f250bc6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25040ae5f0 con 0x7f250406d920 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f250bc6b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250415e830 con 0x7f25040b4660 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f250bc6b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25040ae770 con 0x7f2504147510 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f250406d920 0x7f25041f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f250406d920 0x7f25041f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42156/0 (socket says 172.21.15.105:42156) 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 -- 172.21.15.105:0/4271848269 learned_addr learned my addr 172.21.15.105:0/4271848269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f250a208640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2504147510 0x7f25040b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509206640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25040b4660 0x7f25040b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 -- 172.21.15.105:0/4271848269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2504147510 msgr2=0x7f25040b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2504147510 0x7f25040b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 -- 172.21.15.105:0/4271848269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25040b4660 msgr2=0x7f25040b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25040b4660 0x7f25040b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 -- 172.21.15.105:0/4271848269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2504130ba0 con 0x7f250406d920 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.497+0000 7f2509a07640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f250406d920 0x7f25041f2f00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f250000ba50 tx=0x7f250000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.498+0000 7f24fa7fc640 1 -- 172.21.15.105:0/4271848269 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500019070 con 0x7f250406d920 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.498+0000 7f24fa7fc640 1 -- 172.21.15.105:0/4271848269 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250000cc00 con 0x7f250406d920 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.498+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250412dff0 con 0x7f250406d920 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.498+0000 7f24fa7fc640 1 -- 172.21.15.105:0/4271848269 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500014440 con 0x7f250406d920 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.498+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25041f3440 con 0x7f250406d920 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.499+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2504075120 con 0x7f250406d920 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.499+0000 7f24fa7fc640 1 -- 172.21.15.105:0/4271848269 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f250001c070 con 0x7f250406d920 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.500+0000 7f24fa7fc640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f24e806b500 0x7f24e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.500+0000 7f24fa7fc640 1 -- 172.21.15.105:0/4271848269 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2500089f60 con 0x7f250406d920 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.500+0000 7f2509206640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f24e806b500 0x7f24e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.500+0000 7f2509206640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f24e806b500 0x7f24e806d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24f4036000 tx=0x7f24f403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.504+0000 7f24fa7fc640 1 -- 172.21.15.105:0/4271848269 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2500058de0 con 0x7f250406d920 2024-03-08T08:28:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.674+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2504074800 con 0x7f250406d920 2024-03-08T08:28:13.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.674+0000 7f24fa7fc640 1 -- 172.21.15.105:0/4271848269 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25000586e0 con 0x7f250406d920 2024-03-08T08:28:13.675 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f24e806b500 msgr2=0x7f24e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f24e806b500 0x7f24e806d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24f4036000 tx=0x7f24f403d040 comp rx=0 tx=0).stop 2024-03-08T08:28:13.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f250406d920 msgr2=0x7f25041f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f250406d920 0x7f25041f2f00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f250000ba50 tx=0x7f250000bf10 comp rx=0 tx=0).stop 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 shutdown_connections 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f24e806b500 0x7f24e806d9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2504147510 0x7f25040b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25040b4660 0x7f25040b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 --2- 172.21.15.105:0/4271848269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f250406d920 0x7f25041f2f00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 >> 172.21.15.105:0/4271848269 conn(0x7f25040b1740 msgr2=0x7f25040b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 shutdown_connections 2024-03-08T08:28:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.676+0000 7f250bc6b640 1 -- 172.21.15.105:0/4271848269 wait complete. 2024-03-08T08:28:13.688 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:13.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.790+0000 7f0e7ce9a640 1 Processor -- start 2024-03-08T08:28:13.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.790+0000 7f0e7ce9a640 1 -- start start 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e7ce9a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0e7815aad0 0x7f0e7815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e7ce9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0e7806edf0 0x7f0e7815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e7ce9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7815b360 con 0x7f0e7815aeb0 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e7ce9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7815ebb0 con 0x7f0e7806edf0 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e7ce9a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7815ee90 con 0x7f0e7815aad0 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e76d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0e7806edf0 0x7f0e7815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e75d9b640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0e7815aeb0 0x7f0e7806e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42328/0 (socket says 172.21.15.105:42328) 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e75d9b640 1 -- 172.21.15.105:0/1113199530 learned_addr learned my addr 172.21.15.105:0/1113199530 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.791+0000 7f0e7659c640 1 --2- 172.21.15.105:0/1113199530 >> v2:172.21.15.144:3300/0 conn(0x7f0e7815aad0 0x7f0e7815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.792+0000 7f0e7559a640 1 -- 172.21.15.105:0/1113199530 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428607675 0 0) 0x7f0e7815b360 con 0x7f0e7815aeb0 2024-03-08T08:28:13.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.792+0000 7f0e7559a640 1 -- 172.21.15.105:0/1113199530 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e60003520 con 0x7f0e7815aeb0 2024-03-08T08:28:13.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.792+0000 7f0e76d9d640 1 -- 172.21.15.105:0/1113199530 >> v2:172.21.15.144:3300/0 conn(0x7f0e7815aad0 msgr2=0x7f0e7815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:13.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.792+0000 7f0e76d9d640 1 --2- 172.21.15.105:0/1113199530 >> v2:172.21.15.144:3300/0 conn(0x7f0e7815aad0 0x7f0e7815e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.792+0000 7f0e76d9d640 1 -- 172.21.15.105:0/1113199530 >> v1:172.21.15.105:6789/0 conn(0x7f0e7815aeb0 legacy=0x7f0e7806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.792+0000 7f0e76d9d640 1 -- 172.21.15.105:0/1113199530 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7815f170 con 0x7f0e7806edf0 2024-03-08T08:28:13.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.792+0000 7f0e76d9d640 1 --2- 172.21.15.105:0/1113199530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7806edf0 0x7f0e7815c760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0e7806a520 tx=0x7f0e7002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9f7969ff3d2863 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7559a640 1 -- 172.21.15.105:0/1113199530 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70039070 con 0x7f0e7806edf0 2024-03-08T08:28:13.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7559a640 1 -- 172.21.15.105:0/1113199530 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7002cc80 con 0x7f0e7806edf0 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7559a640 1 -- 172.21.15.105:0/1113199530 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70034660 con 0x7f0e7806edf0 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/1113199530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7806edf0 msgr2=0x7f0e7815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/1113199530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7806edf0 0x7f0e7815c760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0e7806a520 tx=0x7f0e7002bed0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/1113199530 shutdown_connections 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/1113199530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7806edf0 0x7f0e7815c760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/1113199530 >> v2:172.21.15.144:3300/0 conn(0x7f0e7815aad0 0x7f0e7815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/1113199530 >> 172.21.15.105:0/1113199530 conn(0x7f0e7806bd90 msgr2=0x7f0e7806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/1113199530 shutdown_connections 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.793+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/1113199530 wait complete. 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7ce9a640 1 Processor -- start 2024-03-08T08:28:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7ce9a640 1 -- start start 2024-03-08T08:28:13.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7ce9a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e7806edf0 0x7f0e78155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7ce9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7815aad0 0x7f0e7813b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7ce9a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e7815aeb0 0x7f0e7813ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7ce9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780728a0 con 0x7f0e7815aad0 2024-03-08T08:28:13.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7ce9a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7815c240 con 0x7f0e7815aeb0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7ce9a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7815cca0 con 0x7f0e7806edf0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e75d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7815aad0 0x7f0e7813b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e75d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7815aad0 0x7f0e7813b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42178/0 (socket says 172.21.15.105:42178) 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e75d9b640 1 -- 172.21.15.105:0/4293630754 learned_addr learned my addr 172.21.15.105:0/4293630754 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e76d9d640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e7815aeb0 0x7f0e7813ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.795+0000 7f0e7659c640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e7806edf0 0x7f0e78155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e75d9b640 1 -- 172.21.15.105:0/4293630754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e7806edf0 msgr2=0x7f0e78155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e75d9b640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e7806edf0 0x7f0e78155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e75d9b640 1 -- 172.21.15.105:0/4293630754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e7815aeb0 msgr2=0x7f0e7813ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e75d9b640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e7815aeb0 0x7f0e7813ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e75d9b640 1 -- 172.21.15.105:0/4293630754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7813e4e0 con 0x7f0e7815aad0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e75d9b640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7815aad0 0x7f0e7813b4f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c390 tx=0x7f0e6c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e66ffd640 1 -- 172.21.15.105:0/4293630754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c017070 con 0x7f0e7815aad0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e66ffd640 1 -- 172.21.15.105:0/4293630754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c005ce0 con 0x7f0e7815aad0 2024-03-08T08:28:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7813d960 con 0x7f0e7815aad0 2024-03-08T08:28:13.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e66ffd640 1 -- 172.21.15.105:0/4293630754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0072f0 con 0x7f0e7815aad0 2024-03-08T08:28:13.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.796+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7813df20 con 0x7f0e7815aad0 2024-03-08T08:28:13.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.797+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e78075120 con 0x7f0e7815aad0 2024-03-08T08:28:13.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.798+0000 7f0e66ffd640 1 -- 172.21.15.105:0/4293630754 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e6c01a070 con 0x7f0e7815aad0 2024-03-08T08:28:13.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.798+0000 7f0e66ffd640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e5806b4d0 0x7f0e5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:13.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.798+0000 7f0e66ffd640 1 -- 172.21.15.105:0/4293630754 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e6c0894e0 con 0x7f0e7815aad0 2024-03-08T08:28:13.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.798+0000 7f0e7659c640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e5806b4d0 0x7f0e5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:13.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.798+0000 7f0e7659c640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e5806b4d0 0x7f0e5806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009b40 tx=0x7f0e68009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:13.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.802+0000 7f0e66ffd640 1 -- 172.21.15.105:0/4293630754 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e6c057da0 con 0x7f0e7815aad0 2024-03-08T08:28:13.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.971+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e780cd210 con 0x7f0e7815aad0 2024-03-08T08:28:13.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.972+0000 7f0e66ffd640 1 -- 172.21.15.105:0/4293630754 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e6c05be40 con 0x7f0e7815aad0 2024-03-08T08:28:13.973 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e5806b4d0 msgr2=0x7f0e5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e5806b4d0 0x7f0e5806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009b40 tx=0x7f0e68009190 comp rx=0 tx=0).stop 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7815aad0 msgr2=0x7f0e7813b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7815aad0 0x7f0e7813b4f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c390 tx=0x7f0e6c00c850 comp rx=0 tx=0).stop 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 shutdown_connections 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0e5806b4d0 0x7f0e5806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e7815aeb0 0x7f0e7813ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e7815aad0 0x7f0e7813b4f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 --2- 172.21.15.105:0/4293630754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e7806edf0 0x7f0e78155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 >> 172.21.15.105:0/4293630754 conn(0x7f0e7806bd90 msgr2=0x7f0e780cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.973+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 shutdown_connections 2024-03-08T08:28:13.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:13.974+0000 7f0e7ce9a640 1 -- 172.21.15.105:0/4293630754 wait complete. 2024-03-08T08:28:13.985 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:14.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.087+0000 7f290aed4640 1 Processor -- start 2024-03-08T08:28:14.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.087+0000 7f290aed4640 1 -- start start 2024-03-08T08:28:14.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f290aed4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2904161470 0x7f290406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f290aed4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2904161850 0x7f29040d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f290aed4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f290406d370 0x7f290406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f290aed4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29040d2ba0 con 0x7f290406d370 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f290aed4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29040d48a0 con 0x7f2904161470 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f290aed4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29040d4b80 con 0x7f2904161850 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f2909471640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f290406d370 0x7f290406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f2909471640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f290406d370 0x7f290406b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42194/0 (socket says 172.21.15.105:42194) 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.088+0000 7f2909471640 1 -- 172.21.15.105:0/2281634998 learned_addr learned my addr 172.21.15.105:0/2281634998 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2903fff640 1 --2- 172.21.15.105:0/2281634998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2904161850 0x7f29040d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2908c70640 1 --2- 172.21.15.105:0/2281634998 >> v2:172.21.15.144:3300/0 conn(0x7f2904161470 0x7f290406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2909471640 1 -- 172.21.15.105:0/2281634998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2904161850 msgr2=0x7f29040d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2909471640 1 --2- 172.21.15.105:0/2281634998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2904161850 0x7f29040d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2909471640 1 -- 172.21.15.105:0/2281634998 >> v2:172.21.15.144:3300/0 conn(0x7f2904161470 msgr2=0x7f290406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2909471640 1 --2- 172.21.15.105:0/2281634998 >> v2:172.21.15.144:3300/0 conn(0x7f2904161470 0x7f290406c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2909471640 1 -- 172.21.15.105:0/2281634998 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290406c170 con 0x7f290406d370 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2903fff640 1 --2- 172.21.15.105:0/2281634998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2904161850 0x7f29040d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.089+0000 7f2909471640 1 --2- 172.21.15.105:0/2281634998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f290406d370 0x7f290406b5a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f290408c850 tx=0x7f28fc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4425a38875916b75 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f29037fe640 1 -- 172.21.15.105:0/2281634998 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc02c9e0 con 0x7f290406d370 2024-03-08T08:28:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f29037fe640 1 -- 172.21.15.105:0/2281634998 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc02cb80 con 0x7f290406d370 2024-03-08T08:28:14.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f29037fe640 1 -- 172.21.15.105:0/2281634998 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0343f0 con 0x7f290406d370 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 -- 172.21.15.105:0/2281634998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f290406d370 msgr2=0x7f290406b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 --2- 172.21.15.105:0/2281634998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f290406d370 0x7f290406b5a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f290408c850 tx=0x7f28fc02be40 comp rx=0 tx=0).stop 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 -- 172.21.15.105:0/2281634998 shutdown_connections 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 --2- 172.21.15.105:0/2281634998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f290406d370 0x7f290406b5a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 --2- 172.21.15.105:0/2281634998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2904161850 0x7f29040d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 --2- 172.21.15.105:0/2281634998 >> v2:172.21.15.144:3300/0 conn(0x7f2904161470 0x7f290406c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 -- 172.21.15.105:0/2281634998 >> 172.21.15.105:0/2281634998 conn(0x7f29040d3ce0 msgr2=0x7f29040d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 -- 172.21.15.105:0/2281634998 shutdown_connections 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.090+0000 7f290aed4640 1 -- 172.21.15.105:0/2281634998 wait complete. 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.091+0000 7f290aed4640 1 Processor -- start 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.091+0000 7f290aed4640 1 -- start start 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f290aed4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f290406d370 0x7f29040b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f290aed4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2904161470 0x7f29040c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f290aed4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2904161850 0x7f29041e7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f290aed4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290406cf40 con 0x7f2904161470 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f290aed4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290406bae0 con 0x7f2904161850 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f290aed4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29040c8e70 con 0x7f290406d370 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f2903fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2904161470 0x7f29040c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f2903fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2904161470 0x7f29040c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42204/0 (socket says 172.21.15.105:42204) 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.092+0000 7f2903fff640 1 -- 172.21.15.105:0/351811932 learned_addr learned my addr 172.21.15.105:0/351811932 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2909471640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2904161850 0x7f29041e7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2908c70640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f290406d370 0x7f29040b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2903fff640 1 -- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f290406d370 msgr2=0x7f29040b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2903fff640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f290406d370 0x7f29040b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2903fff640 1 -- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2904161850 msgr2=0x7f29041e7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2903fff640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2904161850 0x7f29041e7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2903fff640 1 -- 172.21.15.105:0/351811932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040c9150 con 0x7f2904161470 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2909471640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2904161850 0x7f29041e7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f2903fff640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2904161470 0x7f29040c8090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28f800d7d0 tx=0x7f28f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f29017fa640 1 -- 172.21.15.105:0/351811932 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8004030 con 0x7f2904161470 2024-03-08T08:28:14.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f29017fa640 1 -- 172.21.15.105:0/351811932 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f80041d0 con 0x7f2904161470 2024-03-08T08:28:14.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29040c9430 con 0x7f2904161470 2024-03-08T08:28:14.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f29017fa640 1 -- 172.21.15.105:0/351811932 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f80096a0 con 0x7f2904161470 2024-03-08T08:28:14.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.093+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29040c99f0 con 0x7f2904161470 2024-03-08T08:28:14.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.094+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2904075120 con 0x7f2904161470 2024-03-08T08:28:14.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.094+0000 7f29017fa640 1 -- 172.21.15.105:0/351811932 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28f8009840 con 0x7f2904161470 2024-03-08T08:28:14.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.095+0000 7f29017fa640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28f006b4d0 0x7f28f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.095+0000 7f29017fa640 1 -- 172.21.15.105:0/351811932 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28f808a4f0 con 0x7f2904161470 2024-03-08T08:28:14.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.095+0000 7f2908c70640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28f006b4d0 0x7f28f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.095+0000 7f2908c70640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28f006b4d0 0x7f28f006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28f4006fd0 tx=0x7f28f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.099+0000 7f29017fa640 1 -- 172.21.15.105:0/351811932 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28f8059370 con 0x7f2904161470 2024-03-08T08:28:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.269+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f290415fa40 con 0x7f2904161470 2024-03-08T08:28:14.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.269+0000 7f29017fa640 1 -- 172.21.15.105:0/351811932 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28f8058c70 con 0x7f2904161470 2024-03-08T08:28:14.271 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28f006b4d0 msgr2=0x7f28f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28f006b4d0 0x7f28f006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28f4006fd0 tx=0x7f28f4008040 comp rx=0 tx=0).stop 2024-03-08T08:28:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2904161470 msgr2=0x7f29040c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2904161470 0x7f29040c8090 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28f800d7d0 tx=0x7f28f800dc90 comp rx=0 tx=0).stop 2024-03-08T08:28:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 shutdown_connections 2024-03-08T08:28:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28f006b4d0 0x7f28f006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2904161850 0x7f29041e7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2904161470 0x7f29040c8090 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 --2- 172.21.15.105:0/351811932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f290406d370 0x7f29040b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.271+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 >> 172.21.15.105:0/351811932 conn(0x7f29040d3ce0 msgr2=0x7f290415f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:14.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.272+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 shutdown_connections 2024-03-08T08:28:14.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.272+0000 7f290aed4640 1 -- 172.21.15.105:0/351811932 wait complete. 2024-03-08T08:28:14.283 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T08:28:14.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.383+0000 7f87ff82e640 1 Processor -- start 2024-03-08T08:28:14.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.384+0000 7f87ff82e640 1 -- start start 2024-03-08T08:28:14.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.384+0000 7f87ff82e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 0x7f87f8071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.384+0000 7f87ff82e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87f80b4d00 0x7f87f815af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.384+0000 7f87ff82e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f815b940 con 0x7f87f815b4a0 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.384+0000 7f87ff82e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f815bc20 con 0x7f87f80b4d00 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.384+0000 7f87ff82e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8070840 con 0x7f87f8072440 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87fcdc9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f87f80b4d00 0x7f87f815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87fddcb640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f87f815b4a0 0x7f87f806e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42336/0 (socket says 172.21.15.105:42336) 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87fddcb640 1 -- 172.21.15.105:0/3113705378 learned_addr learned my addr 172.21.15.105:0/3113705378 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87fd5ca640 1 --2- 172.21.15.105:0/3113705378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 0x7f87f8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87fcdc9640 1 -- 172.21.15.105:0/3113705378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 msgr2=0x7f87f8071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87fcdc9640 1 --2- 172.21.15.105:0/3113705378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 0x7f87f8071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87fcdc9640 1 -- 172.21.15.105:0/3113705378 >> v1:172.21.15.105:6789/0 conn(0x7f87f815b4a0 legacy=0x7f87f806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87fcdc9640 1 -- 172.21.15.105:0/3113705378 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f8070b20 con 0x7f87f80b4d00 2024-03-08T08:28:14.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.385+0000 7f87effff640 1 -- 172.21.15.105:0/3113705378 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808918377 0 0) 0x7f87f815b940 con 0x7f87f815b4a0 2024-03-08T08:28:14.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87fcdc9640 1 --2- 172.21.15.105:0/3113705378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b4d00 0x7f87f815af60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f87f40097a0 tx=0x7f87f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5e0099577c81b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87effff640 1 -- 172.21.15.105:0/3113705378 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4035020 con 0x7f87f80b4d00 2024-03-08T08:28:14.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87effff640 1 -- 172.21.15.105:0/3113705378 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4031830 con 0x7f87f80b4d00 2024-03-08T08:28:14.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87effff640 1 -- 172.21.15.105:0/3113705378 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4031c00 con 0x7f87f80b4d00 2024-03-08T08:28:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87ff82e640 1 -- 172.21.15.105:0/3113705378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b4d00 msgr2=0x7f87f815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3113705378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b4d00 0x7f87f815af60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f87f40097a0 tx=0x7f87f402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87ff82e640 1 -- 172.21.15.105:0/3113705378 shutdown_connections 2024-03-08T08:28:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3113705378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f80b4d00 0x7f87f815af60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3113705378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 0x7f87f8071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87ff82e640 1 -- 172.21.15.105:0/3113705378 >> 172.21.15.105:0/3113705378 conn(0x7f87f80b1740 msgr2=0x7f87f80b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87ff82e640 1 -- 172.21.15.105:0/3113705378 shutdown_connections 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.386+0000 7f87ff82e640 1 -- 172.21.15.105:0/3113705378 wait complete. 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.387+0000 7f87ff82e640 1 Processor -- start 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87ff82e640 1 -- start start 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87ff82e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 0x7f87f8157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87ff82e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f80b4d00 0x7f87f8143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87ff82e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f815b4a0 0x7f87f814f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87ff82e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80729a0 con 0x7f87f815b4a0 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87ff82e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8072820 con 0x7f87f80b4d00 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87ff82e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f806e7b0 con 0x7f87f8072440 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87fddcb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f815b4a0 0x7f87f814f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87fddcb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f815b4a0 0x7f87f814f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42226/0 (socket says 172.21.15.105:42226) 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.388+0000 7f87fddcb640 1 -- 172.21.15.105:0/3834596658 learned_addr learned my addr 172.21.15.105:0/3834596658 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:14.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87fd5ca640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 0x7f87f8157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87fcdc9640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f80b4d00 0x7f87f8143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87fddcb640 1 -- 172.21.15.105:0/3834596658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 msgr2=0x7f87f8157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87fddcb640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 0x7f87f8157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87fddcb640 1 -- 172.21.15.105:0/3834596658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f80b4d00 msgr2=0x7f87f8143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87fddcb640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f80b4d00 0x7f87f8143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87fddcb640 1 -- 172.21.15.105:0/3834596658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f81f5c90 con 0x7f87f815b4a0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87fddcb640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f815b4a0 0x7f87f814f580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87f000c6d0 tx=0x7f87f000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87edffb640 1 -- 172.21.15.105:0/3834596658 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0018070 con 0x7f87f815b4a0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87edffb640 1 -- 172.21.15.105:0/3834596658 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f0004d60 con 0x7f87f815b4a0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f8143ff0 con 0x7f87f815b4a0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87edffb640 1 -- 172.21.15.105:0/3834596658 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000a880 con 0x7f87f815b4a0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.389+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f81f5360 con 0x7f87f815b4a0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.390+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f8075120 con 0x7f87f815b4a0 2024-03-08T08:28:14.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.390+0000 7f87edffb640 1 -- 172.21.15.105:0/3834596658 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87f001b070 con 0x7f87f815b4a0 2024-03-08T08:28:14.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.391+0000 7f87edffb640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f87dc06b4d0 0x7f87dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.391+0000 7f87edffb640 1 -- 172.21.15.105:0/3834596658 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87f00899e0 con 0x7f87f815b4a0 2024-03-08T08:28:14.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.391+0000 7f87fd5ca640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f87dc06b4d0 0x7f87dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.391+0000 7f87fd5ca640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f87dc06b4d0 0x7f87dc06d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87e8006fd0 tx=0x7f87e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.395+0000 7f87edffb640 1 -- 172.21.15.105:0/3834596658 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87f0058860 con 0x7f87f815b4a0 2024-03-08T08:28:14.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.580+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f87f8158860 con 0x7f87f815b4a0 2024-03-08T08:28:14.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.581+0000 7f87edffb640 1 -- 172.21.15.105:0/3834596658 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f87f0058160 con 0x7f87f815b4a0 2024-03-08T08:28:14.582 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-08T08:28:14.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f87dc06b4d0 msgr2=0x7f87dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f87dc06b4d0 0x7f87dc06d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87e8006fd0 tx=0x7f87e8008040 comp rx=0 tx=0).stop 2024-03-08T08:28:14.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f815b4a0 msgr2=0x7f87f814f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f815b4a0 0x7f87f814f580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87f000c6d0 tx=0x7f87f000cb90 comp rx=0 tx=0).stop 2024-03-08T08:28:14.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 shutdown_connections 2024-03-08T08:28:14.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f87dc06b4d0 0x7f87dc06d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f815b4a0 0x7f87f814f580 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f80b4d00 0x7f87f8143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 --2- 172.21.15.105:0/3834596658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87f8072440 0x7f87f8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 >> 172.21.15.105:0/3834596658 conn(0x7f87f80b1740 msgr2=0x7f87f80b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:14.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 shutdown_connections 2024-03-08T08:28:14.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.583+0000 7f87ff82e640 1 -- 172.21.15.105:0/3834596658 wait complete. 2024-03-08T08:28:14.595 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-08T08:28:14.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:14.595 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-08T08:28:14.630 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:28:14.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:14.630 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-08T08:28:14.702 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:14.702 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:14.717 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:14.717 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:14.773 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:14.773 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:14.828 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:14.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.965+0000 7f26b7ed4640 1 Processor -- start 2024-03-08T08:28:14.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.966+0000 7f26b7ed4640 1 -- start start 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.966+0000 7f26b7ed4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f26b0072440 0x7f26b0071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.966+0000 7f26b7ed4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26b00b4d00 0x7f26b015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.966+0000 7f26b7ed4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b015b940 con 0x7f26b015b4a0 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.966+0000 7f26b7ed4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b015bc20 con 0x7f26b00b4d00 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.966+0000 7f26b7ed4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b0070840 con 0x7f26b0072440 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.966+0000 7f26b546f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26b00b4d00 0x7f26b015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.967+0000 7f26b546f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f26b00b4d00 0x7f26b015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42228/0 (socket says 172.21.15.105:42228) 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.967+0000 7f26b546f640 1 -- 172.21.15.105:0/214430302 learned_addr learned my addr 172.21.15.105:0/214430302 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.967+0000 7f26b5c70640 1 --2- 172.21.15.105:0/214430302 >> v2:172.21.15.144:3300/0 conn(0x7f26b0072440 0x7f26b0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.967+0000 7f26b546f640 1 -- 172.21.15.105:0/214430302 >> v2:172.21.15.144:3300/0 conn(0x7f26b0072440 msgr2=0x7f26b0071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.967+0000 7f26b546f640 1 --2- 172.21.15.105:0/214430302 >> v2:172.21.15.144:3300/0 conn(0x7f26b0072440 0x7f26b0071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.967+0000 7f26b546f640 1 -- 172.21.15.105:0/214430302 >> v1:172.21.15.144:6789/0 conn(0x7f26b015b4a0 legacy=0x7f26b006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.967+0000 7f26b546f640 1 -- 172.21.15.105:0/214430302 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b0070b20 con 0x7f26b00b4d00 2024-03-08T08:28:14.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.967+0000 7f26b5c70640 1 --2- 172.21.15.105:0/214430302 >> v2:172.21.15.144:3300/0 conn(0x7f26b0072440 0x7f26b0071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b546f640 1 --2- 172.21.15.105:0/214430302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015af60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26b006cae0 tx=0x7f26a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59d3334f0a8ba65 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b4c6e640 1 -- 172.21.15.105:0/214430302 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0035020 con 0x7f26b00b4d00 2024-03-08T08:28:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b4c6e640 1 -- 172.21.15.105:0/214430302 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a0031870 con 0x7f26b00b4d00 2024-03-08T08:28:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b4c6e640 1 -- 172.21.15.105:0/214430302 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0031c40 con 0x7f26b00b4d00 2024-03-08T08:28:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b7ed4640 1 -- 172.21.15.105:0/214430302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 msgr2=0x7f26b015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/214430302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015af60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26b006cae0 tx=0x7f26a002b910 comp rx=0 tx=0).stop 2024-03-08T08:28:14.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b7ed4640 1 -- 172.21.15.105:0/214430302 shutdown_connections 2024-03-08T08:28:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/214430302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015af60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/214430302 >> v2:172.21.15.144:3300/0 conn(0x7f26b0072440 0x7f26b0071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b7ed4640 1 -- 172.21.15.105:0/214430302 >> 172.21.15.105:0/214430302 conn(0x7f26b00b1740 msgr2=0x7f26b00b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b7ed4640 1 -- 172.21.15.105:0/214430302 shutdown_connections 2024-03-08T08:28:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.968+0000 7f26b7ed4640 1 -- 172.21.15.105:0/214430302 wait complete. 2024-03-08T08:28:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.969+0000 7f26b7ed4640 1 Processor -- start 2024-03-08T08:28:14.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b7ed4640 1 -- start start 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b7ed4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b0072440 0x7f26b013f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b7ed4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b7ed4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b015b4a0 0x7f26b01542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b7ed4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b00729a0 con 0x7f26b00b4d00 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b7ed4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b0072820 con 0x7f26b015b4a0 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b7ed4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b006e7b0 con 0x7f26b0072440 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b546f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b546f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42232/0 (socket says 172.21.15.105:42232) 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b546f640 1 -- 172.21.15.105:0/2321923546 learned_addr learned my addr 172.21.15.105:0/2321923546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b6471640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b015b4a0 0x7f26b01542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.970+0000 7f26b5c70640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b0072440 0x7f26b013f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b546f640 1 -- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b0072440 msgr2=0x7f26b013f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b546f640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b0072440 0x7f26b013f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b546f640 1 -- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b015b4a0 msgr2=0x7f26b01542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b546f640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b015b4a0 0x7f26b01542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b546f640 1 -- 172.21.15.105:0/2321923546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b013f8b0 con 0x7f26b00b4d00 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b5c70640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b0072440 0x7f26b013f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b6471640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b015b4a0 0x7f26b01542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b546f640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015ad40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c00 tx=0x7f26a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26a67fc640 1 -- 172.21.15.105:0/2321923546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0035040 con 0x7f26b00b4d00 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26a67fc640 1 -- 172.21.15.105:0/2321923546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a00307d0 con 0x7f26b00b4d00 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b015b280 con 0x7f26b00b4d00 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26a67fc640 1 -- 172.21.15.105:0/2321923546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0030bc0 con 0x7f26b00b4d00 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.971+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b0152820 con 0x7f26b00b4d00 2024-03-08T08:28:14.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.972+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b0075120 con 0x7f26b00b4d00 2024-03-08T08:28:14.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.972+0000 7f26a67fc640 1 -- 172.21.15.105:0/2321923546 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26a004c060 con 0x7f26b00b4d00 2024-03-08T08:28:14.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.973+0000 7f26a67fc640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f269406b4d0 0x7f269406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:14.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.973+0000 7f26a67fc640 1 -- 172.21.15.105:0/2321923546 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26a00ae580 con 0x7f26b00b4d00 2024-03-08T08:28:14.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.973+0000 7f26b5c70640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f269406b4d0 0x7f269406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:14.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.973+0000 7f26b5c70640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f269406b4d0 0x7f269406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00e550 tx=0x7f26ac007ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:14.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:14.977+0000 7f26a67fc640 1 -- 172.21.15.105:0/2321923546 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26a0040070 con 0x7f26b00b4d00 2024-03-08T08:28:15.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.148+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26b00b4240 con 0x7f26b00b4d00 2024-03-08T08:28:15.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.149+0000 7f26a67fc640 1 -- 172.21.15.105:0/2321923546 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f26a007d490 con 0x7f26b00b4d00 2024-03-08T08:28:15.151 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:15.151 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f269406b4d0 msgr2=0x7f269406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f269406b4d0 0x7f269406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00e550 tx=0x7f26ac007ab0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 msgr2=0x7f26b015ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015ad40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c00 tx=0x7f26a00046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 shutdown_connections 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f269406b4d0 0x7f269406d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26b015b4a0 0x7f26b01542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26b00b4d00 0x7f26b015ad40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 --2- 172.21.15.105:0/2321923546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26b0072440 0x7f26b013f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.151+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 >> 172.21.15.105:0/2321923546 conn(0x7f26b00b1740 msgr2=0x7f26b00b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:15.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.152+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 shutdown_connections 2024-03-08T08:28:15.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.152+0000 7f26b7ed4640 1 -- 172.21.15.105:0/2321923546 wait complete. 2024-03-08T08:28:15.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:15.164 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:15.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.265+0000 7f46389a2640 1 Processor -- start 2024-03-08T08:28:15.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.265+0000 7f46389a2640 1 -- start start 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.266+0000 7f46389a2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f463415f000 0x7f463415af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.266+0000 7f46389a2640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f463415b450 0x7f4634158b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.266+0000 7f46389a2640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463415e540 con 0x7f46340b4440 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.266+0000 7f46389a2640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463415ba80 con 0x7f463415b450 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.266+0000 7f46389a2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463415bd60 con 0x7f463415f000 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f4632ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f463415f000 0x7f463415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f46337fe640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f46340b4440 0x7f46340b4820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:35614/0 (socket says 172.21.15.105:35614) 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f46337fe640 1 -- 172.21.15.105:0/770999357 learned_addr learned my addr 172.21.15.105:0/770999357 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f4633fff640 1 --2- 172.21.15.105:0/770999357 >> v2:172.21.15.144:3300/0 conn(0x7f463415b450 0x7f4634158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f46327fc640 1 -- 172.21.15.105:0/770999357 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574053617 0 0) 0x7f463415e540 con 0x7f46340b4440 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f46327fc640 1 -- 172.21.15.105:0/770999357 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4620003520 con 0x7f46340b4440 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f4632ffd640 1 -- 172.21.15.105:0/770999357 >> v2:172.21.15.144:3300/0 conn(0x7f463415b450 msgr2=0x7f4634158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:15.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f4632ffd640 1 --2- 172.21.15.105:0/770999357 >> v2:172.21.15.144:3300/0 conn(0x7f463415b450 0x7f4634158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f4632ffd640 1 -- 172.21.15.105:0/770999357 >> v1:172.21.15.144:6789/0 conn(0x7f46340b4440 legacy=0x7f46340b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.267+0000 7f4632ffd640 1 -- 172.21.15.105:0/770999357 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463415e800 con 0x7f463415f000 2024-03-08T08:28:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f4632ffd640 1 --2- 172.21.15.105:0/770999357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f463415f000 0x7f463415af10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f461c0097a0 tx=0x7f461c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ff05c8e7affadd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46327fc640 1 -- 172.21.15.105:0/770999357 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c035020 con 0x7f463415f000 2024-03-08T08:28:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46327fc640 1 -- 172.21.15.105:0/770999357 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c031830 con 0x7f463415f000 2024-03-08T08:28:15.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46327fc640 1 -- 172.21.15.105:0/770999357 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c031be0 con 0x7f463415f000 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46389a2640 1 -- 172.21.15.105:0/770999357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f463415f000 msgr2=0x7f463415af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46389a2640 1 --2- 172.21.15.105:0/770999357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f463415f000 0x7f463415af10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f461c0097a0 tx=0x7f461c02b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46389a2640 1 -- 172.21.15.105:0/770999357 shutdown_connections 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46389a2640 1 --2- 172.21.15.105:0/770999357 >> v2:172.21.15.144:3300/0 conn(0x7f463415b450 0x7f4634158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46389a2640 1 --2- 172.21.15.105:0/770999357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f463415f000 0x7f463415af10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.268+0000 7f46389a2640 1 -- 172.21.15.105:0/770999357 >> 172.21.15.105:0/770999357 conn(0x7f463406bd90 msgr2=0x7f463406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.269+0000 7f46389a2640 1 -- 172.21.15.105:0/770999357 shutdown_connections 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.269+0000 7f46389a2640 1 -- 172.21.15.105:0/770999357 wait complete. 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.270+0000 7f46389a2640 1 Processor -- start 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.270+0000 7f46389a2640 1 -- start start 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.270+0000 7f46389a2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46340b4440 0x7f463413f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f46389a2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f463415b450 0x7f463413fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f46389a2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f463415f000 0x7f4634155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f46389a2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463406e3e0 con 0x7f463415b450 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f46389a2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463406e260 con 0x7f463415f000 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f46389a2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4634151ca0 con 0x7f46340b4440 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f463415b450 0x7f463413fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f463415b450 0x7f463413fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42244/0 (socket says 172.21.15.105:42244) 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 -- 172.21.15.105:0/3726483838 learned_addr learned my addr 172.21.15.105:0/3726483838 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f46337fe640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46340b4440 0x7f463413f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4633fff640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f463415f000 0x7f4634155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 -- 172.21.15.105:0/3726483838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46340b4440 msgr2=0x7f463413f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46340b4440 0x7f463413f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 -- 172.21.15.105:0/3726483838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f463415f000 msgr2=0x7f4634155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f463415f000 0x7f4634155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 -- 172.21.15.105:0/3726483838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46341531f0 con 0x7f463415b450 2024-03-08T08:28:15.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f4632ffd640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f463415b450 0x7f463413fd60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f462bfff640 1 -- 172.21.15.105:0/3726483838 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0044d0 con 0x7f463415b450 2024-03-08T08:28:15.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f462bfff640 1 -- 172.21.15.105:0/3726483838 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c003610 con 0x7f463415b450 2024-03-08T08:28:15.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f462bfff640 1 -- 172.21.15.105:0/3726483838 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c003a00 con 0x7f463415b450 2024-03-08T08:28:15.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46341534d0 con 0x7f463415b450 2024-03-08T08:28:15.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.271+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4634153a90 con 0x7f463415b450 2024-03-08T08:28:15.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.272+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463404efc0 con 0x7f463415b450 2024-03-08T08:28:15.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.273+0000 7f462bfff640 1 -- 172.21.15.105:0/3726483838 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f461c038020 con 0x7f463415b450 2024-03-08T08:28:15.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.273+0000 7f462bfff640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f461406b670 0x7f461406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.273+0000 7f46337fe640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f461406b670 0x7f461406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.273+0000 7f462bfff640 1 -- 172.21.15.105:0/3726483838 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f461c0ac370 con 0x7f463415b450 2024-03-08T08:28:15.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.274+0000 7f46337fe640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f461406b670 0x7f461406db30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4624005880 tx=0x7f4624009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.278+0000 7f462bfff640 1 -- 172.21.15.105:0/3726483838 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f461c07ac30 con 0x7f463415b450 2024-03-08T08:28:15.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.448+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f463406eee0 con 0x7f463415b450 2024-03-08T08:28:15.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.448+0000 7f462bfff640 1 -- 172.21.15.105:0/3726483838 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f461c07ecd0 con 0x7f463415b450 2024-03-08T08:28:15.449 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f461406b670 msgr2=0x7f461406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f461406b670 0x7f461406db30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4624005880 tx=0x7f4624009040 comp rx=0 tx=0).stop 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f463415b450 msgr2=0x7f463413fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f463415b450 0x7f463413fd60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c0046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 shutdown_connections 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f461406b670 0x7f461406db30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f463415f000 0x7f4634155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f463415b450 0x7f463413fd60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 --2- 172.21.15.105:0/3726483838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46340b4440 0x7f463413f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 >> 172.21.15.105:0/3726483838 conn(0x7f463406bd90 msgr2=0x7f463414ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:15.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 shutdown_connections 2024-03-08T08:28:15.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.450+0000 7f46389a2640 1 -- 172.21.15.105:0/3726483838 wait complete. 2024-03-08T08:28:15.462 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:15.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.562+0000 7fbfa7577640 1 Processor -- start 2024-03-08T08:28:15.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.563+0000 7fbfa7577640 1 -- start start 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.563+0000 7fbfa7577640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfa015dfa0 0x7fbfa006c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.563+0000 7fbfa7577640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa015e380 con 0x7fbfa006b040 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.563+0000 7fbfa7577640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa00b5f60 con 0x7fbfa00ad910 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.563+0000 7fbfa7577640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa00b6240 con 0x7fbfa015dfa0 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.564+0000 7fbfa5313640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfa015dfa0 0x7fbfa006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.564+0000 7fbfa5313640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfa015dfa0 0x7fbfa006c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42252/0 (socket says 172.21.15.105:42252) 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.564+0000 7fbfa5313640 1 -- 172.21.15.105:0/179069321 learned_addr learned my addr 172.21.15.105:0/179069321 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.564+0000 7fbfa5313640 1 -- 172.21.15.105:0/179069321 >> v1:172.21.15.144:6789/0 conn(0x7fbfa00ad910 legacy=0x7fbfa00d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.564+0000 7fbfa5313640 1 -- 172.21.15.105:0/179069321 >> v1:172.21.15.105:6789/0 conn(0x7fbfa006b040 legacy=0x7fbfa00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.564+0000 7fbfa5313640 1 -- 172.21.15.105:0/179069321 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa006b720 con 0x7fbfa015dfa0 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbfa5313640 1 --2- 172.21.15.105:0/179069321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa006c200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009870 tx=0x7fbf9002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f970f83b69b205b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbf97fff640 1 -- 172.21.15.105:0/179069321 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90034070 con 0x7fbfa015dfa0 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbf97fff640 1 -- 172.21.15.105:0/179069321 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf90030860 con 0x7fbfa015dfa0 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbf97fff640 1 -- 172.21.15.105:0/179069321 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90030c30 con 0x7fbfa015dfa0 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbfa7577640 1 -- 172.21.15.105:0/179069321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 msgr2=0x7fbfa006c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbfa7577640 1 --2- 172.21.15.105:0/179069321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa006c200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009870 tx=0x7fbf9002b9a0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbfa7577640 1 -- 172.21.15.105:0/179069321 shutdown_connections 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbfa7577640 1 --2- 172.21.15.105:0/179069321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa006c200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbfa7577640 1 -- 172.21.15.105:0/179069321 >> 172.21.15.105:0/179069321 conn(0x7fbfa00b50f0 msgr2=0x7fbfa00b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbfa7577640 1 -- 172.21.15.105:0/179069321 shutdown_connections 2024-03-08T08:28:15.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.565+0000 7fbfa7577640 1 -- 172.21.15.105:0/179069321 wait complete. 2024-03-08T08:28:15.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.566+0000 7fbfa7577640 1 Processor -- start 2024-03-08T08:28:15.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa7577640 1 -- start start 2024-03-08T08:28:15.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa7577640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa006b040 0x7fbfa0156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa7577640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa00ad910 0x7fbfa01560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa7577640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa0153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa7577640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa006d000 con 0x7fbfa015dfa0 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa7577640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa006bce0 con 0x7fbfa006b040 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa7577640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa006d180 con 0x7fbfa00ad910 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa5b14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa0153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa5b14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa0153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42254/0 (socket says 172.21.15.105:42254) 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa5b14640 1 -- 172.21.15.105:0/3078197937 learned_addr learned my addr 172.21.15.105:0/3078197937 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa5313640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa006b040 0x7fbfa0156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.567+0000 7fbfa4b12640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa00ad910 0x7fbfa01560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbfa5b14640 1 -- 172.21.15.105:0/3078197937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa00ad910 msgr2=0x7fbfa01560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbfa5b14640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa00ad910 0x7fbfa01560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbfa5b14640 1 -- 172.21.15.105:0/3078197937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa006b040 msgr2=0x7fbfa0156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbfa5b14640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa006b040 0x7fbfa0156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbfa5b14640 1 -- 172.21.15.105:0/3078197937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0155320 con 0x7fbfa015dfa0 2024-03-08T08:28:15.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbfa5b14640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa0153770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c7b0 tx=0x7fbf9800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbf95ffb640 1 -- 172.21.15.105:0/3078197937 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98004b60 con 0x7fbfa015dfa0 2024-03-08T08:28:15.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbf95ffb640 1 -- 172.21.15.105:0/3078197937 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf98004d00 con 0x7fbfa015dfa0 2024-03-08T08:28:15.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa01555e0 con 0x7fbfa015dfa0 2024-03-08T08:28:15.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbf95ffb640 1 -- 172.21.15.105:0/3078197937 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800aa90 con 0x7fbfa015dfa0 2024-03-08T08:28:15.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.568+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa0155ba0 con 0x7fbfa015dfa0 2024-03-08T08:28:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.569+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa0155da0 con 0x7fbfa015dfa0 2024-03-08T08:28:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.569+0000 7fbf95ffb640 1 -- 172.21.15.105:0/3078197937 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf9800ac30 con 0x7fbfa015dfa0 2024-03-08T08:28:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.570+0000 7fbf95ffb640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbf8406b4d0 0x7fbf8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.570+0000 7fbf95ffb640 1 -- 172.21.15.105:0/3078197937 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf9808a440 con 0x7fbfa015dfa0 2024-03-08T08:28:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.570+0000 7fbfa5313640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbf8406b4d0 0x7fbf8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.570+0000 7fbfa5313640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbf8406b4d0 0x7fbf8406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf90036000 tx=0x7fbf9003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.574+0000 7fbf95ffb640 1 -- 172.21.15.105:0/3078197937 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfa0155da0 con 0x7fbfa015dfa0 2024-03-08T08:28:15.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.744+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfa0075120 con 0x7fbfa015dfa0 2024-03-08T08:28:15.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.744+0000 7fbf95ffb640 1 -- 172.21.15.105:0/3078197937 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf98020070 con 0x7fbfa015dfa0 2024-03-08T08:28:15.745 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:15.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbf8406b4d0 msgr2=0x7fbf8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbf8406b4d0 0x7fbf8406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbf90036000 tx=0x7fbf9003b040 comp rx=0 tx=0).stop 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 msgr2=0x7fbfa0153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa0153770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c7b0 tx=0x7fbf9800cc70 comp rx=0 tx=0).stop 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 shutdown_connections 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbf8406b4d0 0x7fbf8406d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfa015dfa0 0x7fbfa0153770 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa00ad910 0x7fbfa01560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 --2- 172.21.15.105:0/3078197937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa006b040 0x7fbfa0156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 >> 172.21.15.105:0/3078197937 conn(0x7fbfa00b50f0 msgr2=0x7fbfa0154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 shutdown_connections 2024-03-08T08:28:15.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.746+0000 7fbfa7577640 1 -- 172.21.15.105:0/3078197937 wait complete. 2024-03-08T08:28:15.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:15.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.862+0000 7f027148f640 1 Processor -- start 2024-03-08T08:28:15.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.862+0000 7f027148f640 1 -- start start 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.863+0000 7f027148f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f026c06b390 0x7f026c06c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.863+0000 7f027148f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026c06f830 con 0x7f026c15df20 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.863+0000 7f027148f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026c06fad0 con 0x7f026c0b5990 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.863+0000 7f027148f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026c06dbb0 con 0x7f026c06b390 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.863+0000 7f026a7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f026c06b390 0x7f026c06c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.863+0000 7f026a7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f026c06b390 0x7f026c06c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42270/0 (socket says 172.21.15.105:42270) 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.863+0000 7f026a7fc640 1 -- 172.21.15.105:0/2508863697 learned_addr learned my addr 172.21.15.105:0/2508863697 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.864+0000 7f026a7fc640 1 -- 172.21.15.105:0/2508863697 >> v1:172.21.15.144:6789/0 conn(0x7f026c0b5990 legacy=0x7f026c0ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.864+0000 7f026a7fc640 1 -- 172.21.15.105:0/2508863697 >> v1:172.21.15.105:6789/0 conn(0x7f026c15df20 legacy=0x7f026c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.864+0000 7f026a7fc640 1 -- 172.21.15.105:0/2508863697 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c06fdb0 con 0x7f026c06b390 2024-03-08T08:28:15.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.864+0000 7f026a7fc640 1 --2- 172.21.15.105:0/2508863697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c06b390 0x7f026c06c8b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0260009870 tx=0x7f026002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f9e45e421a763e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.864+0000 7f0269ffb640 1 -- 172.21.15.105:0/2508863697 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260034070 con 0x7f026c06b390 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.864+0000 7f0269ffb640 1 -- 172.21.15.105:0/2508863697 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0260030860 con 0x7f026c06b390 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.864+0000 7f0269ffb640 1 -- 172.21.15.105:0/2508863697 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260030c30 con 0x7f026c06b390 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.865+0000 7f027148f640 1 -- 172.21.15.105:0/2508863697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c06b390 msgr2=0x7f026c06c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.865+0000 7f027148f640 1 --2- 172.21.15.105:0/2508863697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c06b390 0x7f026c06c8b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0260009870 tx=0x7f026002b9a0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.865+0000 7f027148f640 1 -- 172.21.15.105:0/2508863697 shutdown_connections 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.865+0000 7f027148f640 1 --2- 172.21.15.105:0/2508863697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c06b390 0x7f026c06c8b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.865+0000 7f027148f640 1 -- 172.21.15.105:0/2508863697 >> 172.21.15.105:0/2508863697 conn(0x7f026c06bd90 msgr2=0x7f026c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.865+0000 7f027148f640 1 -- 172.21.15.105:0/2508863697 shutdown_connections 2024-03-08T08:28:15.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.865+0000 7f027148f640 1 -- 172.21.15.105:0/2508863697 wait complete. 2024-03-08T08:28:15.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.866+0000 7f027148f640 1 Processor -- start 2024-03-08T08:28:15.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.866+0000 7f027148f640 1 -- start start 2024-03-08T08:28:15.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f027148f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f026c06b390 0x7f026c1492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f027148f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f026c0b5990 0x7f026c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f027148f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c15df20 0x7f026c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f027148f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0ae080 con 0x7f026c15df20 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f027148f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c06e0e0 con 0x7f026c06b390 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f027148f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0ae200 con 0x7f026c0b5990 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c15df20 0x7f026c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c15df20 0x7f026c153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42284/0 (socket says 172.21.15.105:42284) 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026b7fe640 1 -- 172.21.15.105:0/935508873 learned_addr learned my addr 172.21.15.105:0/935508873 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026a7fc640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f026c0b5990 0x7f026c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026affd640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f026c06b390 0x7f026c1492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026b7fe640 1 -- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f026c0b5990 msgr2=0x7f026c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026b7fe640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f026c0b5990 0x7f026c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026b7fe640 1 -- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f026c06b390 msgr2=0x7f026c1492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.867+0000 7f026b7fe640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f026c06b390 0x7f026c1492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:15.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.868+0000 7f026b7fe640 1 -- 172.21.15.105:0/935508873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c1548a0 con 0x7f026c15df20 2024-03-08T08:28:15.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.868+0000 7f026a7fc640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f026c0b5990 0x7f026c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:15.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.868+0000 7f026b7fe640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c15df20 0x7f026c153ee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f026400c400 tx=0x7f026400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.868+0000 7f02537fe640 1 -- 172.21.15.105:0/935508873 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264017070 con 0x7f026c15df20 2024-03-08T08:28:15.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.868+0000 7f02537fe640 1 -- 172.21.15.105:0/935508873 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02640052d0 con 0x7f026c15df20 2024-03-08T08:28:15.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.868+0000 7f02537fe640 1 -- 172.21.15.105:0/935508873 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264003810 con 0x7f026c15df20 2024-03-08T08:28:15.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.868+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c154b80 con 0x7f026c15df20 2024-03-08T08:28:15.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.868+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c155420 con 0x7f026c15df20 2024-03-08T08:28:15.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.869+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026c154d50 con 0x7f026c15df20 2024-03-08T08:28:15.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.869+0000 7f02537fe640 1 -- 172.21.15.105:0/935508873 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f026401a070 con 0x7f026c15df20 2024-03-08T08:28:15.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.870+0000 7f02537fe640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f025806b4d0 0x7f025806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:15.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.870+0000 7f02537fe640 1 -- 172.21.15.105:0/935508873 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02640894d0 con 0x7f026c15df20 2024-03-08T08:28:15.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.870+0000 7f026affd640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f025806b4d0 0x7f025806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:15.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.870+0000 7f026affd640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f025806b4d0 0x7f025806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f025c0012e0 tx=0x7f025c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:15.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:15.874+0000 7f02537fe640 1 -- 172.21.15.105:0/935508873 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02640583e0 con 0x7f026c15df20 2024-03-08T08:28:16.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.045+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f026c075120 con 0x7f026c15df20 2024-03-08T08:28:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.045+0000 7f02537fe640 1 -- 172.21.15.105:0/935508873 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0264087e20 con 0x7f026c15df20 2024-03-08T08:28:16.047 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f025806b4d0 msgr2=0x7f025806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f025806b4d0 0x7f025806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f025c0012e0 tx=0x7f025c009040 comp rx=0 tx=0).stop 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c15df20 msgr2=0x7f026c153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c15df20 0x7f026c153ee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f026400c400 tx=0x7f026400c8c0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 shutdown_connections 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f025806b4d0 0x7f025806d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f026c15df20 0x7f026c153ee0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f026c0b5990 0x7f026c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 --2- 172.21.15.105:0/935508873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f026c06b390 0x7f026c1492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.047+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 >> 172.21.15.105:0/935508873 conn(0x7f026c06bd90 msgr2=0x7f026c0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:16.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.048+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 shutdown_connections 2024-03-08T08:28:16.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.048+0000 7f027148f640 1 -- 172.21.15.105:0/935508873 wait complete. 2024-03-08T08:28:16.059 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T08:28:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.160+0000 7f290330d640 1 Processor -- start 2024-03-08T08:28:16.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.160+0000 7f290330d640 1 -- start start 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f290330d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28fc0cfdb0 0x7f28fc0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f290330d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28fc0d06e0 0x7f28fc15ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f290330d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc0b2c20 con 0x7f28fc0b2740 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f290330d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc0af4c0 con 0x7f28fc0cfdb0 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f290330d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc0af7a0 con 0x7f28fc0d06e0 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f29010a9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28fc0cfdb0 0x7f28fc0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f29010a9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f28fc0cfdb0 0x7f28fc0b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42296/0 (socket says 172.21.15.105:42296) 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f29010a9640 1 -- 172.21.15.105:0/2812872155 learned_addr learned my addr 172.21.15.105:0/2812872155 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f29008a8640 1 --2- 172.21.15.105:0/2812872155 >> v2:172.21.15.144:3300/0 conn(0x7f28fc0d06e0 0x7f28fc15ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f29010a9640 1 -- 172.21.15.105:0/2812872155 >> v2:172.21.15.144:3300/0 conn(0x7f28fc0d06e0 msgr2=0x7f28fc15ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:16.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f29010a9640 1 --2- 172.21.15.105:0/2812872155 >> v2:172.21.15.144:3300/0 conn(0x7f28fc0d06e0 0x7f28fc15ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f29010a9640 1 -- 172.21.15.105:0/2812872155 >> v1:172.21.15.144:6789/0 conn(0x7f28fc0b2740 legacy=0x7f28fc15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:16.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.161+0000 7f29010a9640 1 -- 172.21.15.105:0/2812872155 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0cc490 con 0x7f28fc0cfdb0 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f29010a9640 1 --2- 172.21.15.105:0/2812872155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0cfdb0 0x7f28fc0b1f90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0cf430 tx=0x7f28ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bd7aa04687ab14 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f28f3fff640 1 -- 172.21.15.105:0/2812872155 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec02e070 con 0x7f28fc0cfdb0 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f28f3fff640 1 -- 172.21.15.105:0/2812872155 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec037070 con 0x7f28fc0cfdb0 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f28f3fff640 1 -- 172.21.15.105:0/2812872155 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec031a00 con 0x7f28fc0cfdb0 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f290330d640 1 -- 172.21.15.105:0/2812872155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0cfdb0 msgr2=0x7f28fc0b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f290330d640 1 --2- 172.21.15.105:0/2812872155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0cfdb0 0x7f28fc0b1f90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0cf430 tx=0x7f28ec02b880 comp rx=0 tx=0).stop 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f290330d640 1 -- 172.21.15.105:0/2812872155 shutdown_connections 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f290330d640 1 --2- 172.21.15.105:0/2812872155 >> v2:172.21.15.144:3300/0 conn(0x7f28fc0d06e0 0x7f28fc15ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f290330d640 1 --2- 172.21.15.105:0/2812872155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0cfdb0 0x7f28fc0b1f90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.162+0000 7f290330d640 1 -- 172.21.15.105:0/2812872155 >> 172.21.15.105:0/2812872155 conn(0x7f28fc0b36a0 msgr2=0x7f28fc0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.163+0000 7f290330d640 1 -- 172.21.15.105:0/2812872155 shutdown_connections 2024-03-08T08:28:16.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.163+0000 7f290330d640 1 -- 172.21.15.105:0/2812872155 wait complete. 2024-03-08T08:28:16.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f290330d640 1 Processor -- start 2024-03-08T08:28:16.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f290330d640 1 -- start start 2024-03-08T08:28:16.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f290330d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28fc0b2740 0x7f28fc14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f290330d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28fc0cfdb0 0x7f28fc14d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f290330d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0d06e0 0x7f28fc06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f290330d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc165b70 con 0x7f28fc0d06e0 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f290330d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0d03a0 con 0x7f28fc0cfdb0 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f290330d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc165cf0 con 0x7f28fc0b2740 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f29018aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0d06e0 0x7f28fc06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f29018aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0d06e0 0x7f28fc06e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42302/0 (socket says 172.21.15.105:42302) 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.164+0000 7f29018aa640 1 -- 172.21.15.105:0/1370379353 learned_addr learned my addr 172.21.15.105:0/1370379353 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f29008a8640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28fc0cfdb0 0x7f28fc14d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f29010a9640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28fc0b2740 0x7f28fc14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f29018aa640 1 -- 172.21.15.105:0/1370379353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28fc0b2740 msgr2=0x7f28fc14ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f29018aa640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28fc0b2740 0x7f28fc14ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f29018aa640 1 -- 172.21.15.105:0/1370379353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28fc0cfdb0 msgr2=0x7f28fc14d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f29018aa640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28fc0cfdb0 0x7f28fc14d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f29018aa640 1 -- 172.21.15.105:0/1370379353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc06f390 con 0x7f28fc0d06e0 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f29018aa640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0d06e0 0x7f28fc06e930 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f28f400be60 tx=0x7f28f400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f28f1ffb640 1 -- 172.21.15.105:0/1370379353 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4019070 con 0x7f28fc0d06e0 2024-03-08T08:28:16.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f28f1ffb640 1 -- 172.21.15.105:0/1370379353 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4014440 con 0x7f28fc0d06e0 2024-03-08T08:28:16.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f28f1ffb640 1 -- 172.21.15.105:0/1370379353 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4014870 con 0x7f28fc0d06e0 2024-03-08T08:28:16.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc155060 con 0x7f28fc0d06e0 2024-03-08T08:28:16.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.165+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc14e3c0 con 0x7f28fc0d06e0 2024-03-08T08:28:16.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.166+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc075120 con 0x7f28fc0d06e0 2024-03-08T08:28:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.166+0000 7f28f1ffb640 1 -- 172.21.15.105:0/1370379353 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28f401c070 con 0x7f28fc0d06e0 2024-03-08T08:28:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.167+0000 7f28f1ffb640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28e006b500 0x7f28e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.167+0000 7f28f1ffb640 1 -- 172.21.15.105:0/1370379353 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28f408a0a0 con 0x7f28fc0d06e0 2024-03-08T08:28:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.167+0000 7f29010a9640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28e006b500 0x7f28e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.167+0000 7f29010a9640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28e006b500 0x7f28e006d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28ec036000 tx=0x7f28ec03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.171+0000 7f28f1ffb640 1 -- 172.21.15.105:0/1370379353 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28f4058960 con 0x7f28fc0d06e0 2024-03-08T08:28:16.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.354+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28fc0b0720 con 0x7f28fc0d06e0 2024-03-08T08:28:16.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.354+0000 7f28f1ffb640 1 -- 172.21.15.105:0/1370379353 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f28f405ca00 con 0x7f28fc0d06e0 2024-03-08T08:28:16.355 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-08T08:28:16.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28e006b500 msgr2=0x7f28e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28e006b500 0x7f28e006d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28ec036000 tx=0x7f28ec03d040 comp rx=0 tx=0).stop 2024-03-08T08:28:16.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0d06e0 msgr2=0x7f28fc06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0d06e0 0x7f28fc06e930 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f28f400be60 tx=0x7f28f400bf50 comp rx=0 tx=0).stop 2024-03-08T08:28:16.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 shutdown_connections 2024-03-08T08:28:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f28e006b500 0x7f28e006d9c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28fc0d06e0 0x7f28fc06e930 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28fc0cfdb0 0x7f28fc14d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 --2- 172.21.15.105:0/1370379353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28fc0b2740 0x7f28fc14ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 >> 172.21.15.105:0/1370379353 conn(0x7f28fc0b36a0 msgr2=0x7f28fc15af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 shutdown_connections 2024-03-08T08:28:16.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.356+0000 7f290330d640 1 -- 172.21.15.105:0/1370379353 wait complete. 2024-03-08T08:28:16.368 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-08T08:28:16.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:16.368 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-08T08:28:16.395 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:28:16.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:16.395 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-08T08:28:16.449 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:16.449 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:16.468 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:16.469 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:16.527 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:16.527 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:16.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:16.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.685+0000 7fc8a490d640 1 Processor -- start 2024-03-08T08:28:16.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.685+0000 7fc8a490d640 1 -- start start 2024-03-08T08:28:16.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc8a490d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc8a006b040 0x7fc8a006c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc8a490d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a00ad910 0x7fc8a00d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc8a490d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a015e380 con 0x7fc8a015dfa0 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc8a490d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a00b5f60 con 0x7fc8a006b040 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc8a490d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a00b6240 con 0x7fc8a00ad910 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc89dd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc8a006b040 0x7fc8a006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc89e59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc8a015dfa0 0x7fc8a00ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42368/0 (socket says 172.21.15.105:42368) 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc89e59c640 1 -- 172.21.15.105:0/221227808 learned_addr learned my addr 172.21.15.105:0/221227808 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.686+0000 7fc89ed9d640 1 --2- 172.21.15.105:0/221227808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a00ad910 0x7fc8a00d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.687+0000 7fc89dd9b640 1 -- 172.21.15.105:0/221227808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a00ad910 msgr2=0x7fc8a00d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.687+0000 7fc89dd9b640 1 --2- 172.21.15.105:0/221227808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a00ad910 0x7fc8a00d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.687+0000 7fc89dd9b640 1 -- 172.21.15.105:0/221227808 >> v1:172.21.15.105:6789/0 conn(0x7fc8a015dfa0 legacy=0x7fc8a00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.687+0000 7fc89dd9b640 1 -- 172.21.15.105:0/221227808 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a006b720 con 0x7fc8a006b040 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.687+0000 7fc89d59a640 1 -- 172.21.15.105:0/221227808 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714596239 0 0) 0x7fc8a015e380 con 0x7fc8a015dfa0 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.687+0000 7fc89ed9d640 1 --2- 172.21.15.105:0/221227808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a00ad910 0x7fc8a00d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.687+0000 7fc89dd9b640 1 --2- 172.21.15.105:0/221227808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a006b040 0x7fc8a006c200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc8940097a0 tx=0x7fc89402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158df67f602cd71e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc89d59a640 1 -- 172.21.15.105:0/221227808 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894035020 con 0x7fc8a006b040 2024-03-08T08:28:16.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc89d59a640 1 -- 172.21.15.105:0/221227808 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894031830 con 0x7fc8a006b040 2024-03-08T08:28:16.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc89d59a640 1 -- 172.21.15.105:0/221227808 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894031c00 con 0x7fc8a006b040 2024-03-08T08:28:16.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc8a490d640 1 -- 172.21.15.105:0/221227808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a006b040 msgr2=0x7fc8a006c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc8a490d640 1 --2- 172.21.15.105:0/221227808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a006b040 0x7fc8a006c200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc8940097a0 tx=0x7fc89402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc8a490d640 1 -- 172.21.15.105:0/221227808 shutdown_connections 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc8a490d640 1 --2- 172.21.15.105:0/221227808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a00ad910 0x7fc8a00d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc8a490d640 1 --2- 172.21.15.105:0/221227808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a006b040 0x7fc8a006c200 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc8a490d640 1 -- 172.21.15.105:0/221227808 >> 172.21.15.105:0/221227808 conn(0x7fc8a00b50f0 msgr2=0x7fc8a00b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc8a490d640 1 -- 172.21.15.105:0/221227808 shutdown_connections 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.688+0000 7fc8a490d640 1 -- 172.21.15.105:0/221227808 wait complete. 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.689+0000 7fc8a490d640 1 Processor -- start 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc8a490d640 1 -- start start 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc8a490d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a006b040 0x7fc8a00b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc8a490d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a00ad910 0x7fc8a01613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc8a490d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a015dfa0 0x7fc8a00b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc8a490d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a006d000 con 0x7fc8a015dfa0 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc8a490d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a006bce0 con 0x7fc8a00ad910 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc8a490d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a006d180 con 0x7fc8a006b040 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc89ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a015dfa0 0x7fc8a00b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc89ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a015dfa0 0x7fc8a00b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42320/0 (socket says 172.21.15.105:42320) 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc89ed9d640 1 -- 172.21.15.105:0/2929842745 learned_addr learned my addr 172.21.15.105:0/2929842745 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.690+0000 7fc89e59c640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a006b040 0x7fc8a00b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc89dd9b640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a00ad910 0x7fc8a01613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc89ed9d640 1 -- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a006b040 msgr2=0x7fc8a00b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc89ed9d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a006b040 0x7fc8a00b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc89ed9d640 1 -- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a00ad910 msgr2=0x7fc8a01613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc89ed9d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a00ad910 0x7fc8a01613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc89ed9d640 1 -- 172.21.15.105:0/2929842745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a00b19a0 con 0x7fc8a015dfa0 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc89dd9b640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a00ad910 0x7fc8a01613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc89ed9d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a015dfa0 0x7fc8a00b1270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e7e0 tx=0x7fc89800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc88effd640 1 -- 172.21.15.105:0/2929842745 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89800f040 con 0x7fc8a015dfa0 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc88effd640 1 -- 172.21.15.105:0/2929842745 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc898004030 con 0x7fc8a015dfa0 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a00b1c80 con 0x7fc8a015dfa0 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc88effd640 1 -- 172.21.15.105:0/2929842745 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980043e0 con 0x7fc8a015dfa0 2024-03-08T08:28:16.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.691+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a00b2240 con 0x7fc8a015dfa0 2024-03-08T08:28:16.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.692+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a0075120 con 0x7fc8a015dfa0 2024-03-08T08:28:16.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.692+0000 7fc88effd640 1 -- 172.21.15.105:0/2929842745 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc89800c6d0 con 0x7fc8a015dfa0 2024-03-08T08:28:16.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.693+0000 7fc88effd640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc88006b4d0 0x7fc88006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.693+0000 7fc88effd640 1 -- 172.21.15.105:0/2929842745 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc898089fa0 con 0x7fc8a015dfa0 2024-03-08T08:28:16.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.693+0000 7fc89e59c640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc88006b4d0 0x7fc88006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.693+0000 7fc89e59c640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc88006b4d0 0x7fc88006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc890007ed0 tx=0x7fc890007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.697+0000 7fc88effd640 1 -- 172.21.15.105:0/2929842745 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc898017020 con 0x7fc8a015dfa0 2024-03-08T08:28:16.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.870+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8a0071ae0 con 0x7fc8a015dfa0 2024-03-08T08:28:16.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.871+0000 7fc88effd640 1 -- 172.21.15.105:0/2929842745 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fc898058d90 con 0x7fc8a015dfa0 2024-03-08T08:28:16.872 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:16.872 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc88006b4d0 msgr2=0x7fc88006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc88006b4d0 0x7fc88006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc890007ed0 tx=0x7fc890007e60 comp rx=0 tx=0).stop 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a015dfa0 msgr2=0x7fc8a00b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a015dfa0 0x7fc8a00b1270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e7e0 tx=0x7fc89800eca0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 shutdown_connections 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc88006b4d0 0x7fc88006d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a015dfa0 0x7fc8a00b1270 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a00ad910 0x7fc8a01613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 --2- 172.21.15.105:0/2929842745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8a006b040 0x7fc8a00b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 >> 172.21.15.105:0/2929842745 conn(0x7fc8a00b50f0 msgr2=0x7fc8a00ad200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 shutdown_connections 2024-03-08T08:28:16.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.873+0000 7fc8a490d640 1 -- 172.21.15.105:0/2929842745 wait complete. 2024-03-08T08:28:16.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:16.885 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:16.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.986+0000 7f0dd869a640 1 Processor -- start 2024-03-08T08:28:16.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.986+0000 7f0dd869a640 1 -- start start 2024-03-08T08:28:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.987+0000 7f0dd869a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0dd006e0a0 0x7f0dd006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.987+0000 7f0dd869a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd015b670 con 0x7f0dd00b4d00 2024-03-08T08:28:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.987+0000 7f0dd869a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd015b940 con 0x7f0dd0072440 2024-03-08T08:28:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.987+0000 7f0dd869a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd006ea50 con 0x7f0dd006e0a0 2024-03-08T08:28:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.987+0000 7f0dd5c35640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0dd00b4d00 0x7f0dd015af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42380/0 (socket says 172.21.15.105:42380) 2024-03-08T08:28:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.987+0000 7f0dd5c35640 1 -- 172.21.15.105:0/1641170973 learned_addr learned my addr 172.21.15.105:0/1641170973 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6c37640 1 --2- 172.21.15.105:0/1641170973 >> v2:172.21.15.144:3300/0 conn(0x7f0dd006e0a0 0x7f0dd006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647470947 0 0) 0x7f0dd015b670 con 0x7f0dd00b4d00 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dbc003520 con 0x7f0dd00b4d00 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 587668038 0 0) 0x7f0dbc003520 con 0x7f0dd00b4d00 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 >> v2:172.21.15.144:3300/0 conn(0x7f0dd006e0a0 msgr2=0x7f0dd006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 --2- 172.21.15.105:0/1641170973 >> v2:172.21.15.144:3300/0 conn(0x7f0dd006e0a0 0x7f0dd006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 >> v1:172.21.15.144:6789/0 conn(0x7f0dd0072440 legacy=0x7f0dd0071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd015bc20 con 0x7f0dd00b4d00 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f0dc0001670 con 0x7f0dd00b4d00 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 >> v1:172.21.15.105:6789/0 conn(0x7f0dd00b4d00 legacy=0x7f0dd015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc00c030 con 0x7f0dbc0039b0 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc00c1e0 con 0x7f0dbc004800 2024-03-08T08:28:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc00c3e0 con 0x7f0dbc0051c0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6436640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd5c35640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6c37640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6436640 1 -- 172.21.15.105:0/1641170973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dbc0051c0 msgr2=0x7f0dbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6436640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dbc0051c0 0x7f0dbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6436640 1 -- 172.21.15.105:0/1641170973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dbc004800 msgr2=0x7f0dbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6436640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dbc004800 0x7f0dbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6436640 1 -- 172.21.15.105:0/1641170973 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc003520 con 0x7f0dbc0039b0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.988+0000 7f0dd6c37640 1 -- 172.21.15.105:0/1641170973 reap_dead start 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.989+0000 7f0dd6436640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8001e50 tx=0x7f0dbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a690a9f62d334f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.989+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8001a70 con 0x7f0dbc0039b0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.989+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8003b70 con 0x7f0dbc0039b0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.989+0000 7f0dd5434640 1 -- 172.21.15.105:0/1641170973 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80050d0 con 0x7f0dbc0039b0 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.990+0000 7f0dd869a640 1 -- 172.21.15.105:0/1641170973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 msgr2=0x7f0dbc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.990+0000 7f0dd869a640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8001e50 tx=0x7f0dbc005830 comp rx=0 tx=0).stop 2024-03-08T08:28:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.990+0000 7f0dd869a640 1 -- 172.21.15.105:0/1641170973 shutdown_connections 2024-03-08T08:28:16.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.990+0000 7f0dd869a640 1 --2- 172.21.15.105:0/1641170973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dbc003dd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.990+0000 7f0dd869a640 1 -- 172.21.15.105:0/1641170973 >> 172.21.15.105:0/1641170973 conn(0x7f0dd00b1740 msgr2=0x7f0dd00b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:16.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.990+0000 7f0dd869a640 1 -- 172.21.15.105:0/1641170973 shutdown_connections 2024-03-08T08:28:16.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.990+0000 7f0dd869a640 1 -- 172.21.15.105:0/1641170973 wait complete. 2024-03-08T08:28:16.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.991+0000 7f0dd869a640 1 Processor -- start 2024-03-08T08:28:16.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.991+0000 7f0dd869a640 1 -- start start 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd869a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dd014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd869a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dd013d650 0x7f0dd00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd869a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dd013dad0 0x7f0dd00c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd869a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0072820 con 0x7f0dbc0039b0 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd869a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0070b20 con 0x7f0dd013dad0 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd869a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd00729a0 con 0x7f0dd013d650 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dd014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dd014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42340/0 (socket says 172.21.15.105:42340) 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 -- 172.21.15.105:0/590016751 learned_addr learned my addr 172.21.15.105:0/590016751 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd5c35640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dd013d650 0x7f0dd00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6c37640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dd013dad0 0x7f0dd00c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 -- 172.21.15.105:0/590016751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dd013d650 msgr2=0x7f0dd00c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dd013d650 0x7f0dd00c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 -- 172.21.15.105:0/590016751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dd013dad0 msgr2=0x7f0dd00c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dd013dad0 0x7f0dd00c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 -- 172.21.15.105:0/590016751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd0149630 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd6436640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dd014a6d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8000f80 tx=0x7f0dc80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dc6ffd640 1 -- 172.21.15.105:0/590016751 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8003970 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dc6ffd640 1 -- 172.21.15.105:0/590016751 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8006b50 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd0149910 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dc6ffd640 1 -- 172.21.15.105:0/590016751 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8007020 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.992+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd0149ed0 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.993+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd014ac10 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.994+0000 7f0dc6ffd640 1 -- 172.21.15.105:0/590016751 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dc8049060 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.994+0000 7f0dc6ffd640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0db406b4d0 0x7f0db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.994+0000 7f0dc6ffd640 1 -- 172.21.15.105:0/590016751 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0dc80ab670 con 0x7f0dbc0039b0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.994+0000 7f0dd5c35640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0db406b4d0 0x7f0db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:16.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.995+0000 7f0dd5c35640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0db406b4d0 0x7f0db406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0dd014af60 tx=0x7f0dc0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:16.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:16.998+0000 7f0dc6ffd640 1 -- 172.21.15.105:0/590016751 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dc8033020 con 0x7f0dbc0039b0 2024-03-08T08:28:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.169+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0dd0075120 con 0x7f0dbc0039b0 2024-03-08T08:28:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.169+0000 7f0dc6ffd640 1 -- 172.21.15.105:0/590016751 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0dc807a4f0 con 0x7f0dbc0039b0 2024-03-08T08:28:17.170 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:17.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0db406b4d0 msgr2=0x7f0db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0db406b4d0 0x7f0db406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0dd014af60 tx=0x7f0dc0009040 comp rx=0 tx=0).stop 2024-03-08T08:28:17.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 msgr2=0x7f0dd014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dd014a6d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8000f80 tx=0x7f0dc80014a0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 shutdown_connections 2024-03-08T08:28:17.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f0db406b4d0 0x7f0db406d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dd013dad0 0x7f0dd00c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0dd013d650 0x7f0dd00c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 --2- 172.21.15.105:0/590016751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0dbc0039b0 0x7f0dd014a6d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 >> 172.21.15.105:0/590016751 conn(0x7f0dd00b1740 msgr2=0x7f0dd00ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 shutdown_connections 2024-03-08T08:28:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.171+0000 7f0dd869a640 1 -- 172.21.15.105:0/590016751 wait complete. 2024-03-08T08:28:17.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:17.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.285+0000 7fd45a635640 1 Processor -- start 2024-03-08T08:28:17.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.285+0000 7fd45a635640 1 -- start start 2024-03-08T08:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.285+0000 7fd45a635640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.285+0000 7fd45a635640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45415b670 con 0x7fd4540b4d00 2024-03-08T08:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.285+0000 7fd45a635640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45415b940 con 0x7fd45406e0a0 2024-03-08T08:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.285+0000 7fd45a635640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45415bc20 con 0x7fd454072440 2024-03-08T08:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd4537fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd4540b4d00 0x7fd45415af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42394/0 (socket says 172.21.15.105:42394) 2024-03-08T08:28:17.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd4537fe640 1 -- 172.21.15.105:0/1811091528 learned_addr learned my addr 172.21.15.105:0/1811091528 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd453fff640 1 --2- 172.21.15.105:0/1811091528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd452ffd640 1 -- 172.21.15.105:0/1811091528 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776718343 0 0) 0x7fd45415b670 con 0x7fd4540b4d00 2024-03-08T08:28:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd452ffd640 1 -- 172.21.15.105:0/1811091528 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd43c003520 con 0x7fd4540b4d00 2024-03-08T08:28:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd453fff640 1 -- 172.21.15.105:0/1811091528 >> v1:172.21.15.144:6789/0 conn(0x7fd45406e0a0 legacy=0x7fd4540cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd452ffd640 1 -- 172.21.15.105:0/1811091528 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725852982 0 0) 0x7fd45415b940 con 0x7fd45406e0a0 2024-03-08T08:28:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd453fff640 1 -- 172.21.15.105:0/1811091528 >> v1:172.21.15.105:6789/0 conn(0x7fd4540b4d00 legacy=0x7fd45415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.286+0000 7fd453fff640 1 -- 172.21.15.105:0/1811091528 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454070840 con 0x7fd454072440 2024-03-08T08:28:17.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.287+0000 7fd453fff640 1 --2- 172.21.15.105:0/1811091528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454071020 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4440097d0 tx=0x7fd44402b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b3ce2888790b97 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.287+0000 7fd452ffd640 1 -- 172.21.15.105:0/1811091528 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444037070 con 0x7fd454072440 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.287+0000 7fd452ffd640 1 -- 172.21.15.105:0/1811091528 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd444030510 con 0x7fd454072440 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.287+0000 7fd452ffd640 1 -- 172.21.15.105:0/1811091528 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448001670 con 0x7fd454072440 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.288+0000 7fd45a635640 1 -- 172.21.15.105:0/1811091528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 msgr2=0x7fd454071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.288+0000 7fd45a635640 1 --2- 172.21.15.105:0/1811091528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454071020 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4440097d0 tx=0x7fd44402b730 comp rx=0 tx=0).stop 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.288+0000 7fd45a635640 1 -- 172.21.15.105:0/1811091528 shutdown_connections 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.288+0000 7fd45a635640 1 --2- 172.21.15.105:0/1811091528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454071020 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.288+0000 7fd45a635640 1 -- 172.21.15.105:0/1811091528 >> 172.21.15.105:0/1811091528 conn(0x7fd4540b1740 msgr2=0x7fd4540b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.288+0000 7fd45a635640 1 -- 172.21.15.105:0/1811091528 shutdown_connections 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.288+0000 7fd45a635640 1 -- 172.21.15.105:0/1811091528 wait complete. 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.289+0000 7fd45a635640 1 Processor -- start 2024-03-08T08:28:17.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.289+0000 7fd45a635640 1 -- start start 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd45a635640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45406e0a0 0x7fd4541541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd45a635640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd45a635640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4540b4d00 0x7fd4541527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd45a635640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd454072820 con 0x7fd45406e0a0 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd45a635640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd454071560 con 0x7fd4540b4d00 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd45a635640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540729a0 con 0x7fd454072440 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45406e0a0 0x7fd4541541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45406e0a0 0x7fd4541541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42344/0 (socket says 172.21.15.105:42344) 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 -- 172.21.15.105:0/4258635789 learned_addr learned my addr 172.21.15.105:0/4258635789 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:17.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd4537fe640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 -- 172.21.15.105:0/4258635789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 msgr2=0x7fd454152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 -- 172.21.15.105:0/4258635789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4540b4d00 msgr2=0x7fd4541527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd458bd2640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4540b4d00 0x7fd4541527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4540b4d00 0x7fd4541527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 -- 172.21.15.105:0/4258635789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454149720 con 0x7fd45406e0a0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd453fff640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45406e0a0 0x7fd4541541a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd444009720 tx=0x7fd4440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd450ff9640 1 -- 172.21.15.105:0/4258635789 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444042070 con 0x7fd45406e0a0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd450ff9640 1 -- 172.21.15.105:0/4258635789 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd444004220 con 0x7fd45406e0a0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd450ff9640 1 -- 172.21.15.105:0/4258635789 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444037040 con 0x7fd45406e0a0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.290+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd454149a00 con 0x7fd45406e0a0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.291+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd454149fc0 con 0x7fd45406e0a0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.291+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd454153e00 con 0x7fd45406e0a0 2024-03-08T08:28:17.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.292+0000 7fd450ff9640 1 -- 172.21.15.105:0/4258635789 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd44403a040 con 0x7fd45406e0a0 2024-03-08T08:28:17.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.292+0000 7fd450ff9640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd44006b4d0 0x7fd44006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.292+0000 7fd450ff9640 1 -- 172.21.15.105:0/4258635789 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4440abb40 con 0x7fd45406e0a0 2024-03-08T08:28:17.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.292+0000 7fd4537fe640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd44006b4d0 0x7fd44006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.293+0000 7fd4537fe640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd44006b4d0 0x7fd44006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4480017e0 tx=0x7fd448009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.296+0000 7fd450ff9640 1 -- 172.21.15.105:0/4258635789 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd44407a9c0 con 0x7fd45406e0a0 2024-03-08T08:28:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.465+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd454075120 con 0x7fd45406e0a0 2024-03-08T08:28:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.466+0000 7fd450ff9640 1 -- 172.21.15.105:0/4258635789 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd44407a2c0 con 0x7fd45406e0a0 2024-03-08T08:28:17.467 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.467+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd44006b4d0 msgr2=0x7fd44006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.467+0000 7fd45a635640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd44006b4d0 0x7fd44006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4480017e0 tx=0x7fd448009040 comp rx=0 tx=0).stop 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.467+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45406e0a0 msgr2=0x7fd4541541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.467+0000 7fd45a635640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45406e0a0 0x7fd4541541a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd444009720 tx=0x7fd4440046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.467+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 shutdown_connections 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.467+0000 7fd45a635640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd44006b4d0 0x7fd44006d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.468+0000 7fd45a635640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4540b4d00 0x7fd4541527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.468+0000 7fd45a635640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd454072440 0x7fd454152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.468+0000 7fd45a635640 1 --2- 172.21.15.105:0/4258635789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd45406e0a0 0x7fd4541541a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.468+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 >> 172.21.15.105:0/4258635789 conn(0x7fd4540b1740 msgr2=0x7fd4540ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.468+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 shutdown_connections 2024-03-08T08:28:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.468+0000 7fd45a635640 1 -- 172.21.15.105:0/4258635789 wait complete. 2024-03-08T08:28:17.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:17.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.581+0000 7feb14a37640 1 Processor -- start 2024-03-08T08:28:17.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.581+0000 7feb14a37640 1 -- start start 2024-03-08T08:28:17.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb14a37640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feb10161520 0x7feb100b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb14a37640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feb100ce330 0x7feb100ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb14a37640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb100b29e0 con 0x7feb100b14f0 2024-03-08T08:28:17.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb14a37640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb100d1930 con 0x7feb100ce330 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb14a37640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb1006c770 con 0x7feb10161520 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb0dd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feb100ce330 0x7feb100ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb0ed9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feb100b14f0 0x7feb100b1960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42396/0 (socket says 172.21.15.105:42396) 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb0ed9d640 1 -- 172.21.15.105:0/3710813685 learned_addr learned my addr 172.21.15.105:0/3710813685 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.582+0000 7feb0e59c640 1 --2- 172.21.15.105:0/3710813685 >> v2:172.21.15.144:3300/0 conn(0x7feb10161520 0x7feb100b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0dd9b640 1 -- 172.21.15.105:0/3710813685 >> v2:172.21.15.144:3300/0 conn(0x7feb10161520 msgr2=0x7feb100b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0dd9b640 1 --2- 172.21.15.105:0/3710813685 >> v2:172.21.15.144:3300/0 conn(0x7feb10161520 0x7feb100b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0dd9b640 1 -- 172.21.15.105:0/3710813685 >> v1:172.21.15.105:6789/0 conn(0x7feb100b14f0 legacy=0x7feb100b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0dd9b640 1 -- 172.21.15.105:0/3710813685 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100d1650 con 0x7feb100ce330 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0d59a640 1 -- 172.21.15.105:0/3710813685 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185414646 0 0) 0x7feb100b29e0 con 0x7feb100b14f0 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0e59c640 1 --2- 172.21.15.105:0/3710813685 >> v2:172.21.15.144:3300/0 conn(0x7feb10161520 0x7feb100b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0dd9b640 1 --2- 172.21.15.105:0/3710813685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb100ce7c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb100cf030 tx=0x7feb0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4148df7f29aadc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0d59a640 1 -- 172.21.15.105:0/3710813685 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04035020 con 0x7feb100ce330 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0d59a640 1 -- 172.21.15.105:0/3710813685 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb04031830 con 0x7feb100ce330 2024-03-08T08:28:17.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.583+0000 7feb0d59a640 1 -- 172.21.15.105:0/3710813685 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04031c00 con 0x7feb100ce330 2024-03-08T08:28:17.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.584+0000 7feb14a37640 1 -- 172.21.15.105:0/3710813685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 msgr2=0x7feb100ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.584+0000 7feb14a37640 1 --2- 172.21.15.105:0/3710813685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb100ce7c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb100cf030 tx=0x7feb0402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.584+0000 7feb14a37640 1 -- 172.21.15.105:0/3710813685 shutdown_connections 2024-03-08T08:28:17.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.584+0000 7feb14a37640 1 --2- 172.21.15.105:0/3710813685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb100ce7c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.584+0000 7feb14a37640 1 --2- 172.21.15.105:0/3710813685 >> v2:172.21.15.144:3300/0 conn(0x7feb10161520 0x7feb100b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.584+0000 7feb14a37640 1 -- 172.21.15.105:0/3710813685 >> 172.21.15.105:0/3710813685 conn(0x7feb100b36a0 msgr2=0x7feb100b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.584+0000 7feb14a37640 1 -- 172.21.15.105:0/3710813685 shutdown_connections 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.584+0000 7feb14a37640 1 -- 172.21.15.105:0/3710813685 wait complete. 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.585+0000 7feb14a37640 1 Processor -- start 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.585+0000 7feb14a37640 1 -- start start 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb14a37640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb100b14f0 0x7feb101f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb14a37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb10151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb14a37640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb10161520 0x7feb100b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb14a37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100b32d0 con 0x7feb100ce330 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb14a37640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1006ca40 con 0x7feb100b14f0 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb14a37640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1006ced0 con 0x7feb10161520 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb0dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb10151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb0dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb10151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42352/0 (socket says 172.21.15.105:42352) 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb0dd9b640 1 -- 172.21.15.105:0/3754885889 learned_addr learned my addr 172.21.15.105:0/3754885889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb0ed9d640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb10161520 0x7feb100b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.586+0000 7feb0e59c640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb100b14f0 0x7feb101f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb0dd9b640 1 -- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb10161520 msgr2=0x7feb100b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb0dd9b640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb10161520 0x7feb100b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb0dd9b640 1 -- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb100b14f0 msgr2=0x7feb101f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb0dd9b640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb100b14f0 0x7feb101f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb0dd9b640 1 -- 172.21.15.105:0/3754885889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1015b080 con 0x7feb100ce330 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb0e59c640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb100b14f0 0x7feb101f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb0dd9b640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb10151e20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feb04009890 tx=0x7feb040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feafeffd640 1 -- 172.21.15.105:0/3754885889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04041070 con 0x7feb100ce330 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feafeffd640 1 -- 172.21.15.105:0/3754885889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0403aa30 con 0x7feb100ce330 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb10152360 con 0x7feb100ce330 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feafeffd640 1 -- 172.21.15.105:0/3754885889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04035040 con 0x7feb100ce330 2024-03-08T08:28:17.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.587+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb100c6a50 con 0x7feb100ce330 2024-03-08T08:28:17.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.588+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb10075120 con 0x7feb100ce330 2024-03-08T08:28:17.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.588+0000 7feafeffd640 1 -- 172.21.15.105:0/3754885889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb04038020 con 0x7feb100ce330 2024-03-08T08:28:17.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.589+0000 7feafeffd640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feaf006b4d0 0x7feaf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.589+0000 7feafeffd640 1 -- 172.21.15.105:0/3754885889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb040abb90 con 0x7feb100ce330 2024-03-08T08:28:17.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.589+0000 7feb0e59c640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feaf006b4d0 0x7feaf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.589+0000 7feb0e59c640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feaf006b4d0 0x7feaf006d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb00003fd0 tx=0x7feb00003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.593+0000 7feafeffd640 1 -- 172.21.15.105:0/3754885889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb0407aa10 con 0x7feb100ce330 2024-03-08T08:28:17.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.762+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb1006dd80 con 0x7feb100ce330 2024-03-08T08:28:17.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.762+0000 7feafeffd640 1 -- 172.21.15.105:0/3754885889 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7feb0407a310 con 0x7feb100ce330 2024-03-08T08:28:17.764 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:17.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feaf006b4d0 msgr2=0x7feaf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feaf006b4d0 0x7feaf006d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb00003fd0 tx=0x7feb00003990 comp rx=0 tx=0).stop 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 msgr2=0x7feb10151e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb10151e20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feb04009890 tx=0x7feb040046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 shutdown_connections 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feaf006b4d0 0x7feaf006d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb10161520 0x7feb100b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb100ce330 0x7feb10151e20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 --2- 172.21.15.105:0/3754885889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb100b14f0 0x7feb101f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 >> 172.21.15.105:0/3754885889 conn(0x7feb100b36a0 msgr2=0x7feb100ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 shutdown_connections 2024-03-08T08:28:17.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.764+0000 7feb14a37640 1 -- 172.21.15.105:0/3754885889 wait complete. 2024-03-08T08:28:17.776 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T08:28:17.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.876+0000 7f9173ced640 1 Processor -- start 2024-03-08T08:28:17.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.876+0000 7f9173ced640 1 -- start start 2024-03-08T08:28:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.877+0000 7f9173ced640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.877+0000 7f9173ced640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f916c0ce790 0x7f916c0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.877+0000 7f9173ced640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0b29e0 con 0x7f916c0ce330 2024-03-08T08:28:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.877+0000 7f9173ced640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0d1930 con 0x7f916c0ce790 2024-03-08T08:28:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.877+0000 7f9173ced640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c06c770 con 0x7f916c161520 2024-03-08T08:28:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.877+0000 7f9171a89640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.877+0000 7f9171a89640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:41304/0 (socket says 172.21.15.105:41304) 2024-03-08T08:28:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.877+0000 7f9171a89640 1 -- 172.21.15.105:0/3863137855 learned_addr learned my addr 172.21.15.105:0/3863137855 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.878+0000 7f917228a640 1 --2- 172.21.15.105:0/3863137855 >> v2:172.21.15.144:3300/0 conn(0x7f916c0ce790 0x7f916c0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.878+0000 7f9171a89640 1 -- 172.21.15.105:0/3863137855 >> v2:172.21.15.144:3300/0 conn(0x7f916c0ce790 msgr2=0x7f916c0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.878+0000 7f9170a87640 1 -- 172.21.15.105:0/3863137855 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682404183 0 0) 0x7f916c0b29e0 con 0x7f916c0ce330 2024-03-08T08:28:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.878+0000 7f9171a89640 1 --2- 172.21.15.105:0/3863137855 >> v2:172.21.15.144:3300/0 conn(0x7f916c0ce790 0x7f916c0b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.878+0000 7f9171a89640 1 -- 172.21.15.105:0/3863137855 >> v1:172.21.15.144:6789/0 conn(0x7f916c0ce330 legacy=0x7f916c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.878+0000 7f9171a89640 1 -- 172.21.15.105:0/3863137855 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0d1650 con 0x7f916c161520 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9171a89640 1 --2- 172.21.15.105:0/3863137855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0b2d90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9168009870 tx=0x7f916802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627978a09fdb3db server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9170a87640 1 -- 172.21.15.105:0/3863137855 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168034070 con 0x7f916c161520 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9170a87640 1 -- 172.21.15.105:0/3863137855 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168030840 con 0x7f916c161520 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9170a87640 1 -- 172.21.15.105:0/3863137855 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168030c10 con 0x7f916c161520 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9173ced640 1 -- 172.21.15.105:0/3863137855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 msgr2=0x7f916c0b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9173ced640 1 --2- 172.21.15.105:0/3863137855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0b2d90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9168009870 tx=0x7f916802b980 comp rx=0 tx=0).stop 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9173ced640 1 -- 172.21.15.105:0/3863137855 shutdown_connections 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9173ced640 1 --2- 172.21.15.105:0/3863137855 >> v2:172.21.15.144:3300/0 conn(0x7f916c0ce790 0x7f916c0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9173ced640 1 --2- 172.21.15.105:0/3863137855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0b2d90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9173ced640 1 -- 172.21.15.105:0/3863137855 >> 172.21.15.105:0/3863137855 conn(0x7f916c0b36a0 msgr2=0x7f916c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9173ced640 1 -- 172.21.15.105:0/3863137855 shutdown_connections 2024-03-08T08:28:17.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.879+0000 7f9173ced640 1 -- 172.21.15.105:0/3863137855 wait complete. 2024-03-08T08:28:17.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9173ced640 1 Processor -- start 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9173ced640 1 -- start start 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9173ced640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c0ce330 0x7f916c0b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9173ced640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c0ce790 0x7f916c0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9173ced640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9173ced640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c161900 con 0x7f916c0ce790 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9173ced640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c06ca40 con 0x7f916c0ce330 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9173ced640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b32d0 con 0x7f916c161520 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9171288640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c0ce790 0x7f916c0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9171288640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c0ce790 0x7f916c0c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42354/0 (socket says 172.21.15.105:42354) 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.881+0000 7f9171288640 1 -- 172.21.15.105:0/3363521986 learned_addr learned my addr 172.21.15.105:0/3363521986 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f917228a640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9171a89640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c0ce330 0x7f916c0b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9171288640 1 -- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 msgr2=0x7f916c0c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9171288640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9171288640 1 -- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c0ce330 msgr2=0x7f916c0b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9171288640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c0ce330 0x7f916c0b6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9171288640 1 -- 172.21.15.105:0/3363521986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0cb890 con 0x7f916c0ce790 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9171a89640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c0ce330 0x7f916c0b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9171288640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c0ce790 0x7f916c0c61e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c460 tx=0x7f915c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f91627fc640 1 -- 172.21.15.105:0/3363521986 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c005480 con 0x7f916c0ce790 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f91627fc640 1 -- 172.21.15.105:0/3363521986 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c0035f0 con 0x7f916c0ce790 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0cbb70 con 0x7f916c0ce790 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f91627fc640 1 -- 172.21.15.105:0/3363521986 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c003a00 con 0x7f916c0ce790 2024-03-08T08:28:17.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.882+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0af220 con 0x7f916c0ce790 2024-03-08T08:28:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.883+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0b3a80 con 0x7f916c0ce790 2024-03-08T08:28:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.883+0000 7f91627fc640 1 -- 172.21.15.105:0/3363521986 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f915c003ba0 con 0x7f916c0ce790 2024-03-08T08:28:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.884+0000 7f91627fc640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f915006b6a0 0x7f915006db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.884+0000 7f9171a89640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f915006b6a0 0x7f915006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.884+0000 7f91627fc640 1 -- 172.21.15.105:0/3363521986 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f915c089de0 con 0x7f916c0ce790 2024-03-08T08:28:17.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.884+0000 7f9171a89640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f915006b6a0 0x7f915006db60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9168036000 tx=0x7f9168008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:17.888+0000 7f91627fc640 1 -- 172.21.15.105:0/3363521986 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f916c0b3a80 con 0x7f916c0ce790 2024-03-08T08:28:18.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.071+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f916c075120 con 0x7f916c0ce790 2024-03-08T08:28:18.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.071+0000 7f91627fc640 1 -- 172.21.15.105:0/3363521986 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f915c058c60 con 0x7f916c0ce790 2024-03-08T08:28:18.072 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-08T08:28:18.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f915006b6a0 msgr2=0x7f915006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f915006b6a0 0x7f915006db60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9168036000 tx=0x7f9168008030 comp rx=0 tx=0).stop 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c0ce790 msgr2=0x7f916c0c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c0ce790 0x7f916c0c61e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c460 tx=0x7f915c00c920 comp rx=0 tx=0).stop 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 shutdown_connections 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f915006b6a0 0x7f915006db60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916c161520 0x7f916c0c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c0ce790 0x7f916c0c61e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 --2- 172.21.15.105:0/3363521986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c0ce330 0x7f916c0b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 >> 172.21.15.105:0/3363521986 conn(0x7f916c0b36a0 msgr2=0x7f916c1607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 shutdown_connections 2024-03-08T08:28:18.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.073+0000 7f9173ced640 1 -- 172.21.15.105:0/3363521986 wait complete. 2024-03-08T08:28:18.085 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-08T08:28:18.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:18.085 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-08T08:28:18.119 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:28:18.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:18.119 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-08T08:28:18.176 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-08T08:28:18.177 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:28:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.276+0000 7f4e8e23b640 1 Processor -- start 2024-03-08T08:28:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.276+0000 7f4e8e23b640 1 -- start start 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e8e23b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e881584b0 0x7f4e880b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e8e23b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4e8815ece0 0x7f4e8806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e8e23b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8815f170 con 0x7f4e880b2e80 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e8e23b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e880af8b0 con 0x7f4e8815ece0 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e8e23b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880ad910 con 0x7f4e881584b0 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e87fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4e8815ece0 0x7f4e8806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e877fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4e880b2e80 0x7f4e880b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42406/0 (socket says 172.21.15.105:42406) 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e877fe640 1 -- 172.21.15.105:0/75416142 learned_addr learned my addr 172.21.15.105:0/75416142 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e86ffd640 1 --2- 172.21.15.105:0/75416142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e881584b0 0x7f4e880b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e87fff640 1 -- 172.21.15.105:0/75416142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e881584b0 msgr2=0x7f4e880b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e87fff640 1 --2- 172.21.15.105:0/75416142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e881584b0 0x7f4e880b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e87fff640 1 -- 172.21.15.105:0/75416142 >> v1:172.21.15.105:6789/0 conn(0x7f4e880b2e80 legacy=0x7f4e880b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e87fff640 1 -- 172.21.15.105:0/75416142 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880adbf0 con 0x7f4e8815ece0 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.277+0000 7f4e867fc640 1 -- 172.21.15.105:0/75416142 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941168780 0 0) 0x7f4e8815f170 con 0x7f4e880b2e80 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e87fff640 1 --2- 172.21.15.105:0/75416142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e8815ece0 0x7f4e8806cd30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009d90 tx=0x7f4e8002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276a94ceeef59ca server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e867fc640 1 -- 172.21.15.105:0/75416142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80039070 con 0x7f4e8815ece0 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e867fc640 1 -- 172.21.15.105:0/75416142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8002cc50 con 0x7f4e8815ece0 2024-03-08T08:28:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e867fc640 1 -- 172.21.15.105:0/75416142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80034440 con 0x7f4e8815ece0 2024-03-08T08:28:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e8e23b640 1 -- 172.21.15.105:0/75416142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e8815ece0 msgr2=0x7f4e8806cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/75416142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e8815ece0 0x7f4e8806cd30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009d90 tx=0x7f4e8002bea0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e8e23b640 1 -- 172.21.15.105:0/75416142 shutdown_connections 2024-03-08T08:28:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/75416142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e8815ece0 0x7f4e8806cd30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/75416142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e881584b0 0x7f4e880b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e8e23b640 1 -- 172.21.15.105:0/75416142 >> 172.21.15.105:0/75416142 conn(0x7f4e880cfdd0 msgr2=0x7f4e880d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e8e23b640 1 -- 172.21.15.105:0/75416142 shutdown_connections 2024-03-08T08:28:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.278+0000 7f4e8e23b640 1 -- 172.21.15.105:0/75416142 wait complete. 2024-03-08T08:28:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e8e23b640 1 Processor -- start 2024-03-08T08:28:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e8e23b640 1 -- start start 2024-03-08T08:28:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e8e23b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e880b2e80 0x7f4e8814c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e8e23b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e881584b0 0x7f4e881e8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e8e23b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8815ece0 0x7f4e881f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e8e23b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880ae490 con 0x7f4e880b2e80 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e8e23b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880ae1b0 con 0x7f4e881584b0 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e8e23b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880ae9b0 con 0x7f4e8815ece0 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.280+0000 7f4e877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e880b2e80 0x7f4e8814c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e880b2e80 0x7f4e8814c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42364/0 (socket says 172.21.15.105:42364) 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e877fe640 1 -- 172.21.15.105:0/1543102633 learned_addr learned my addr 172.21.15.105:0/1543102633 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e87fff640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8815ece0 0x7f4e881f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e877fe640 1 -- 172.21.15.105:0/1543102633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8815ece0 msgr2=0x7f4e881f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e877fe640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8815ece0 0x7f4e881f3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e86ffd640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e881584b0 0x7f4e881e8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e877fe640 1 -- 172.21.15.105:0/1543102633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e881584b0 msgr2=0x7f4e881e8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e877fe640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e881584b0 0x7f4e881e8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e877fe640 1 -- 172.21.15.105:0/1543102633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8814c6b0 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e877fe640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e880b2e80 0x7f4e8814c030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c460 tx=0x7f4e7c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e6ffff640 1 -- 172.21.15.105:0/1543102633 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c018070 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e6ffff640 1 -- 172.21.15.105:0/1543102633 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c007500 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8814c990 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e6ffff640 1 -- 172.21.15.105:0/1543102633 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c004a20 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.281+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8814cf50 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.282+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e88075120 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.283+0000 7f4e6ffff640 1 -- 172.21.15.105:0/1543102633 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e7c01b070 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.283+0000 7f4e6ffff640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e7406b4d0 0x7f4e7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.283+0000 7f4e6ffff640 1 -- 172.21.15.105:0/1543102633 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e7c089560 con 0x7f4e880b2e80 2024-03-08T08:28:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.283+0000 7f4e86ffd640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e7406b4d0 0x7f4e7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.283+0000 7f4e86ffd640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e7406b4d0 0x7f4e7406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e78006fd0 tx=0x7f4e78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.287+0000 7f4e6ffff640 1 -- 172.21.15.105:0/1543102633 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e7c058350 con 0x7f4e880b2e80 2024-03-08T08:28:18.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.496+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e880af250 con 0x7f4e880b2e80 2024-03-08T08:28:18.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.496+0000 7f4e6ffff640 1 -- 172.21.15.105:0/1543102633 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4944 (secure 0 0 0) 0x7f4e7c057c50 con 0x7f4e880b2e80 2024-03-08T08:28:18.498 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:18.498 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:27:38.998370+0000","modified":"2024-03-08T08:27:41.001533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T08:28:18.498 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e7406b4d0 msgr2=0x7f4e7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e7406b4d0 0x7f4e7406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e78006fd0 tx=0x7f4e78008040 comp rx=0 tx=0).stop 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e880b2e80 msgr2=0x7f4e8814c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e880b2e80 0x7f4e8814c030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c460 tx=0x7f4e7c00c920 comp rx=0 tx=0).stop 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 shutdown_connections 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e7406b4d0 0x7f4e7406d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8815ece0 0x7f4e881f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e881584b0 0x7f4e881e8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 --2- 172.21.15.105:0/1543102633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e880b2e80 0x7f4e8814c030 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.498+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 >> 172.21.15.105:0/1543102633 conn(0x7f4e880cfdd0 msgr2=0x7f4e880afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.499+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 shutdown_connections 2024-03-08T08:28:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.499+0000 7f4e8e23b640 1 -- 172.21.15.105:0/1543102633 wait complete. 2024-03-08T08:28:18.511 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:18.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.610+0000 7fba31777640 1 Processor -- start 2024-03-08T08:28:18.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.610+0000 7fba31777640 1 -- start start 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.610+0000 7fba31777640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 0x7fba2c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.610+0000 7fba31777640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fba2c161850 0x7fba2c0d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.610+0000 7fba31777640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c0d48a0 con 0x7fba2c06d370 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.610+0000 7fba31777640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c0d2ba0 con 0x7fba2c161850 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.610+0000 7fba31777640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0d4b80 con 0x7fba2c161470 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.610+0000 7fba2a7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fba2c161850 0x7fba2c0d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.611+0000 7fba2a7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fba2c161850 0x7fba2c0d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42368/0 (socket says 172.21.15.105:42368) 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.611+0000 7fba2a7fc640 1 -- 172.21.15.105:0/238341188 learned_addr learned my addr 172.21.15.105:0/238341188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.611+0000 7fba2affd640 1 --2- 172.21.15.105:0/238341188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 0x7fba2c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.611+0000 7fba2a7fc640 1 -- 172.21.15.105:0/238341188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 msgr2=0x7fba2c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.611+0000 7fba2a7fc640 1 --2- 172.21.15.105:0/238341188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 0x7fba2c06c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.611+0000 7fba2a7fc640 1 -- 172.21.15.105:0/238341188 >> v1:172.21.15.144:6789/0 conn(0x7fba2c06d370 legacy=0x7fba2c06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.611+0000 7fba2a7fc640 1 -- 172.21.15.105:0/238341188 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c06c170 con 0x7fba2c161850 2024-03-08T08:28:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba2a7fc640 1 --2- 172.21.15.105:0/238341188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0d2660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fba20009870 tx=0x7fba2002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6092eaf998b743ed server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba29ffb640 1 -- 172.21.15.105:0/238341188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20035020 con 0x7fba2c161850 2024-03-08T08:28:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba29ffb640 1 -- 172.21.15.105:0/238341188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba20031640 con 0x7fba2c161850 2024-03-08T08:28:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba29ffb640 1 -- 172.21.15.105:0/238341188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2002d040 con 0x7fba2c161850 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba31777640 1 -- 172.21.15.105:0/238341188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 msgr2=0x7fba2c0d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba31777640 1 --2- 172.21.15.105:0/238341188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0d2660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fba20009870 tx=0x7fba2002b8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba31777640 1 -- 172.21.15.105:0/238341188 shutdown_connections 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba31777640 1 --2- 172.21.15.105:0/238341188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0d2660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba31777640 1 --2- 172.21.15.105:0/238341188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 0x7fba2c06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba31777640 1 -- 172.21.15.105:0/238341188 >> 172.21.15.105:0/238341188 conn(0x7fba2c0d3ce0 msgr2=0x7fba2c0d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba31777640 1 -- 172.21.15.105:0/238341188 shutdown_connections 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.612+0000 7fba31777640 1 -- 172.21.15.105:0/238341188 wait complete. 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba31777640 1 Processor -- start 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba31777640 1 -- start start 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba31777640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c06d370 0x7fba2c0b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba31777640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 0x7fba2c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba31777640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba31777640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c06cf40 con 0x7fba2c161850 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba31777640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c06bcb0 con 0x7fba2c06d370 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba31777640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0b9520 con 0x7fba2c161470 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba2b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba2b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42382/0 (socket says 172.21.15.105:42382) 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.614+0000 7fba2b7fe640 1 -- 172.21.15.105:0/1850808216 learned_addr learned my addr 172.21.15.105:0/1850808216 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba2a7fc640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 0x7fba2c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba2affd640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c06d370 0x7fba2c0b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba2b7fe640 1 -- 172.21.15.105:0/1850808216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 msgr2=0x7fba2c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba2b7fe640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 0x7fba2c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba2b7fe640 1 -- 172.21.15.105:0/1850808216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c06d370 msgr2=0x7fba2c0b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba2b7fe640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c06d370 0x7fba2c0b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba2b7fe640 1 -- 172.21.15.105:0/1850808216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c0b8910 con 0x7fba2c161850 2024-03-08T08:28:18.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba2b7fe640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0cc440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c3c0 tx=0x7fba2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba137fe640 1 -- 172.21.15.105:0/1850808216 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240052d0 con 0x7fba2c161850 2024-03-08T08:28:18.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba137fe640 1 -- 172.21.15.105:0/1850808216 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba24005490 con 0x7fba2c161850 2024-03-08T08:28:18.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c0b8bf0 con 0x7fba2c161850 2024-03-08T08:28:18.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba137fe640 1 -- 172.21.15.105:0/1850808216 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24007260 con 0x7fba2c161850 2024-03-08T08:28:18.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.615+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c0b91b0 con 0x7fba2c161850 2024-03-08T08:28:18.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.616+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c075120 con 0x7fba2c161850 2024-03-08T08:28:18.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.617+0000 7fba137fe640 1 -- 172.21.15.105:0/1850808216 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba24007400 con 0x7fba2c161850 2024-03-08T08:28:18.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.617+0000 7fba137fe640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fba1806b4d0 0x7fba1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.617+0000 7fba2affd640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fba1806b4d0 0x7fba1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.617+0000 7fba137fe640 1 -- 172.21.15.105:0/1850808216 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba24089500 con 0x7fba2c161850 2024-03-08T08:28:18.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.617+0000 7fba2affd640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fba1806b4d0 0x7fba1806d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0b9d90 tx=0x7fba1c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.621+0000 7fba137fe640 1 -- 172.21.15.105:0/1850808216 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba24057dc0 con 0x7fba2c161850 2024-03-08T08:28:18.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.794+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba2c0d1bc0 con 0x7fba2c161850 2024-03-08T08:28:18.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.795+0000 7fba137fe640 1 -- 172.21.15.105:0/1850808216 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fba2405be60 con 0x7fba2c161850 2024-03-08T08:28:18.797 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:18.797 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:27:40.659515+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fba1806b4d0 msgr2=0x7fba1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fba1806b4d0 0x7fba1806d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0b9d90 tx=0x7fba1c00b040 comp rx=0 tx=0).stop 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 msgr2=0x7fba2c0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0cc440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c3c0 tx=0x7fba2400c880 comp rx=0 tx=0).stop 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 shutdown_connections 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fba1806b4d0 0x7fba1806d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c161850 0x7fba2c0cc440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fba2c161470 0x7fba2c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 --2- 172.21.15.105:0/1850808216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c06d370 0x7fba2c0b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 >> 172.21.15.105:0/1850808216 conn(0x7fba2c0d3ce0 msgr2=0x7fba2c06b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 shutdown_connections 2024-03-08T08:28:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.797+0000 7fba31777640 1 -- 172.21.15.105:0/1850808216 wait complete. 2024-03-08T08:28:18.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:27:37.021001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:27:37.970196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:18.811 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:18.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.912+0000 7fe768725640 1 Processor -- start 2024-03-08T08:28:18.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.913+0000 7fe768725640 1 -- start start 2024-03-08T08:28:18.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.913+0000 7fe768725640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe76015e340 0x7fe76006b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.913+0000 7fe768725640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe760066990 0x7fe7600d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.913+0000 7fe768725640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe7600d05f0 0x7fe760161c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.913+0000 7fe768725640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76015f020 con 0x7fe760066990 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.913+0000 7fe768725640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76015e720 con 0x7fe7600d05f0 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.913+0000 7fe768725640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe760066ec0 con 0x7fe76015e340 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe765cc0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe760066990 0x7fe7600d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe765cc0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe760066990 0x7fe7600d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42398/0 (socket says 172.21.15.105:42398) 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe765cc0640 1 -- 172.21.15.105:0/1364937254 learned_addr learned my addr 172.21.15.105:0/1364937254 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe7664c1640 1 --2- 172.21.15.105:0/1364937254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe76015e340 0x7fe76006b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe766cc2640 1 --2- 172.21.15.105:0/1364937254 >> v2:172.21.15.144:3300/0 conn(0x7fe7600d05f0 0x7fe760161c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe765cc0640 1 -- 172.21.15.105:0/1364937254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe76015e340 msgr2=0x7fe76006b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe765cc0640 1 --2- 172.21.15.105:0/1364937254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe76015e340 0x7fe76006b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe765cc0640 1 -- 172.21.15.105:0/1364937254 >> v2:172.21.15.144:3300/0 conn(0x7fe7600d05f0 msgr2=0x7fe760161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe765cc0640 1 --2- 172.21.15.105:0/1364937254 >> v2:172.21.15.144:3300/0 conn(0x7fe7600d05f0 0x7fe760161c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe765cc0640 1 -- 172.21.15.105:0/1364937254 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76015e9d0 con 0x7fe760066990 2024-03-08T08:28:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.914+0000 7fe7664c1640 1 --2- 172.21.15.105:0/1364937254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe76015e340 0x7fe76006b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe765cc0640 1 --2- 172.21.15.105:0/1364937254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe760066990 0x7fe7600d00b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe750009870 tx=0x7fe75002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d019614da4242ed9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe7654bf640 1 -- 172.21.15.105:0/1364937254 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750038070 con 0x7fe760066990 2024-03-08T08:28:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe7654bf640 1 -- 172.21.15.105:0/1364937254 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe750031740 con 0x7fe760066990 2024-03-08T08:28:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe7654bf640 1 -- 172.21.15.105:0/1364937254 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75002f050 con 0x7fe760066990 2024-03-08T08:28:18.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 -- 172.21.15.105:0/1364937254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe760066990 msgr2=0x7fe7600d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 --2- 172.21.15.105:0/1364937254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe760066990 0x7fe7600d00b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe750009870 tx=0x7fe75002b7e0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 -- 172.21.15.105:0/1364937254 shutdown_connections 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 --2- 172.21.15.105:0/1364937254 >> v2:172.21.15.144:3300/0 conn(0x7fe7600d05f0 0x7fe760161c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 --2- 172.21.15.105:0/1364937254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe760066990 0x7fe7600d00b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 --2- 172.21.15.105:0/1364937254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe76015e340 0x7fe76006b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 -- 172.21.15.105:0/1364937254 >> 172.21.15.105:0/1364937254 conn(0x7fe760072110 msgr2=0x7fe760072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 -- 172.21.15.105:0/1364937254 shutdown_connections 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.915+0000 7fe768725640 1 -- 172.21.15.105:0/1364937254 wait complete. 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.916+0000 7fe768725640 1 Processor -- start 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe768725640 1 -- start start 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe768725640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe760066990 0x7fe7601e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe768725640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7600d05f0 0x7fe7601f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe768725640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76015e340 0x7fe76006df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe768725640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76015f4b0 con 0x7fe76015e340 2024-03-08T08:28:18.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe768725640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600af530 con 0x7fe7600d05f0 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe768725640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7600b0020 con 0x7fe760066990 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe766cc2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76015e340 0x7fe76006df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe766cc2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76015e340 0x7fe76006df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42404/0 (socket says 172.21.15.105:42404) 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.917+0000 7fe766cc2640 1 -- 172.21.15.105:0/1560919863 learned_addr learned my addr 172.21.15.105:0/1560919863 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe766cc2640 1 -- 172.21.15.105:0/1560919863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe760066990 msgr2=0x7fe7601e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe7664c1640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe760066990 0x7fe7601e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe765cc0640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7600d05f0 0x7fe7601f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe766cc2640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe760066990 0x7fe7601e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe766cc2640 1 -- 172.21.15.105:0/1560919863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7600d05f0 msgr2=0x7fe7601f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe766cc2640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7600d05f0 0x7fe7601f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe766cc2640 1 -- 172.21.15.105:0/1560919863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600b0300 con 0x7fe76015e340 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe766cc2640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76015e340 0x7fe76006df80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe758007e60 tx=0x7fe75800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe756ffd640 1 -- 172.21.15.105:0/1560919863 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe758014620 con 0x7fe76015e340 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe756ffd640 1 -- 172.21.15.105:0/1560919863 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe758019070 con 0x7fe76015e340 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe756ffd640 1 -- 172.21.15.105:0/1560919863 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe758014a20 con 0x7fe76015e340 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7600b05e0 con 0x7fe76015e340 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.918+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7600b0ba0 con 0x7fe76015e340 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.919+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe760075120 con 0x7fe76015e340 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.919+0000 7fe756ffd640 1 -- 172.21.15.105:0/1560919863 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe758014bc0 con 0x7fe76015e340 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.920+0000 7fe756ffd640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe74406b4d0 0x7fe74406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.920+0000 7fe756ffd640 1 -- 172.21.15.105:0/1560919863 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe75808aa90 con 0x7fe76015e340 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.920+0000 7fe7664c1640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe74406b4d0 0x7fe74406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:18.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.920+0000 7fe7664c1640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe74406b4d0 0x7fe74406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0096d0 tx=0x7fe75c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:18.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:18.924+0000 7fe756ffd640 1 -- 172.21.15.105:0/1560919863 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe758059910 con 0x7fe76015e340 2024-03-08T08:28:19.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.091+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe760072e90 con 0x7fe76015e340 2024-03-08T08:28:19.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.092+0000 7fe756ffd640 1 -- 172.21.15.105:0/1560919863 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe758059210 con 0x7fe76015e340 2024-03-08T08:28:19.093 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe74406b4d0 msgr2=0x7fe74406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe74406b4d0 0x7fe74406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0096d0 tx=0x7fe75c008040 comp rx=0 tx=0).stop 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76015e340 msgr2=0x7fe76006df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76015e340 0x7fe76006df80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe758007e60 tx=0x7fe75800e4b0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 shutdown_connections 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe74406b4d0 0x7fe74406d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe76015e340 0x7fe76006df80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7600d05f0 0x7fe7601f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 --2- 172.21.15.105:0/1560919863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe760066990 0x7fe7601e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 >> 172.21.15.105:0/1560919863 conn(0x7fe760072110 msgr2=0x7fe7600c63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:19.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 shutdown_connections 2024-03-08T08:28:19.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.094+0000 7fe768725640 1 -- 172.21.15.105:0/1560919863 wait complete. 2024-03-08T08:28:19.106 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:28:19.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.208+0000 7ff6a82fb640 1 Processor -- start 2024-03-08T08:28:19.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.208+0000 7ff6a82fb640 1 -- start start 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a82fb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff6a00b4000 0x7ff6a015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a82fb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6a006cf90 0x7ff6a00cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a82fb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a015b700 con 0x7ff6a00cf290 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a82fb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a00cd9e0 con 0x7ff6a006cf90 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a82fb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a0072120 con 0x7ff6a00b4000 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a6898640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6a006cf90 0x7ff6a00cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a6898640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6a006cf90 0x7ff6a00cde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42414/0 (socket says 172.21.15.105:42414) 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a6898640 1 -- 172.21.15.105:0/2891766924 learned_addr learned my addr 172.21.15.105:0/2891766924 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:19.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a6097640 1 --2- 172.21.15.105:0/2891766924 >> v2:172.21.15.144:3300/0 conn(0x7ff6a00b4000 0x7ff6a015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a6898640 1 -- 172.21.15.105:0/2891766924 >> v2:172.21.15.144:3300/0 conn(0x7ff6a00b4000 msgr2=0x7ff6a015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a6898640 1 --2- 172.21.15.105:0/2891766924 >> v2:172.21.15.144:3300/0 conn(0x7ff6a00b4000 0x7ff6a015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a6898640 1 -- 172.21.15.105:0/2891766924 >> v1:172.21.15.144:6789/0 conn(0x7ff6a00cf290 legacy=0x7ff6a00b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.209+0000 7ff6a6898640 1 -- 172.21.15.105:0/2891766924 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0072400 con 0x7ff6a006cf90 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.210+0000 7ff6a6898640 1 --2- 172.21.15.105:0/2891766924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a006cf90 0x7ff6a00cde70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0070f60 tx=0x7ff69802e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a28da2f7777c9e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.210+0000 7ff6a5095640 1 -- 172.21.15.105:0/2891766924 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698034070 con 0x7ff6a006cf90 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.210+0000 7ff6a5095640 1 -- 172.21.15.105:0/2891766924 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff698038400 con 0x7ff6a006cf90 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.210+0000 7ff6a5095640 1 -- 172.21.15.105:0/2891766924 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980387d0 con 0x7ff6a006cf90 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.211+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2891766924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a006cf90 msgr2=0x7ff6a00cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.211+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2891766924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a006cf90 0x7ff6a00cde70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0070f60 tx=0x7ff69802e0f0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.211+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2891766924 shutdown_connections 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.211+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2891766924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a006cf90 0x7ff6a00cde70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.211+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2891766924 >> v2:172.21.15.144:3300/0 conn(0x7ff6a00b4000 0x7ff6a015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.211+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2891766924 >> 172.21.15.105:0/2891766924 conn(0x7ff6a00b1740 msgr2=0x7ff6a00b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:19.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.211+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2891766924 shutdown_connections 2024-03-08T08:28:19.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.211+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2891766924 wait complete. 2024-03-08T08:28:19.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.212+0000 7ff6a82fb640 1 Processor -- start 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.212+0000 7ff6a82fb640 1 -- start start 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a82fb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6a006cf90 0x7ff6a014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a82fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a00b4000 0x7ff6a01574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a82fb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6a00cf290 0x7ff6a0158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a82fb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a015a500 con 0x7ff6a00b4000 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a82fb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00726e0 con 0x7ff6a006cf90 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a82fb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a006d420 con 0x7ff6a00cf290 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a00b4000 0x7ff6a01574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a00b4000 0x7ff6a01574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42426/0 (socket says 172.21.15.105:42426) 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 -- 172.21.15.105:0/2265904914 learned_addr learned my addr 172.21.15.105:0/2265904914 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a6898640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6a00cf290 0x7ff6a0158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a6097640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6a006cf90 0x7ff6a014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 -- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6a00cf290 msgr2=0x7ff6a0158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6a00cf290 0x7ff6a0158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 -- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6a006cf90 msgr2=0x7ff6a014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6a006cf90 0x7ff6a014ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 -- 172.21.15.105:0/2265904914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a01569f0 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a6097640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6a006cf90 0x7ff6a014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a5896640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a00b4000 0x7ff6a01574c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff69000b3f0 tx=0x7ff69000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff696ffd640 1 -- 172.21.15.105:0/2265904914 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690015020 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff696ffd640 1 -- 172.21.15.105:0/2265904914 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff690011830 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a0156cd0 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff696ffd640 1 -- 172.21.15.105:0/2265904914 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690011c20 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.213+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a0159350 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.214+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a0075120 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.215+0000 7ff696ffd640 1 -- 172.21.15.105:0/2265904914 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff690018020 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.215+0000 7ff696ffd640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff68406b4d0 0x7ff68406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.215+0000 7ff696ffd640 1 -- 172.21.15.105:0/2265904914 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff690089e10 con 0x7ff6a00b4000 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.215+0000 7ff6a6097640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff68406b4d0 0x7ff68406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.216+0000 7ff6a6097640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff68406b4d0 0x7ff68406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00a9d0 tx=0x7ff69c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.219+0000 7ff696ffd640 1 -- 172.21.15.105:0/2265904914 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff69001f030 con 0x7ff6a00b4000 2024-03-08T08:28:19.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.389+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6a0071390 con 0x7ff6a00b4000 2024-03-08T08:28:19.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.389+0000 7ff696ffd640 1 -- 172.21.15.105:0/2265904914 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff690058c90 con 0x7ff6a00b4000 2024-03-08T08:28:19.390 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:28:19.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff68406b4d0 msgr2=0x7ff68406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff68406b4d0 0x7ff68406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00a9d0 tx=0x7ff69c008040 comp rx=0 tx=0).stop 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a00b4000 msgr2=0x7ff6a01574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a00b4000 0x7ff6a01574c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff69000b3f0 tx=0x7ff69000b8b0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 shutdown_connections 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff68406b4d0 0x7ff68406d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6a00cf290 0x7ff6a0158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6a00b4000 0x7ff6a01574c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 --2- 172.21.15.105:0/2265904914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6a006cf90 0x7ff6a014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 >> 172.21.15.105:0/2265904914 conn(0x7ff6a00b1740 msgr2=0x7ff6a015be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 shutdown_connections 2024-03-08T08:28:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.391+0000 7ff6a82fb640 1 -- 172.21.15.105:0/2265904914 wait complete. 2024-03-08T08:28:19.403 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:28:19.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.505+0000 7ff943119640 1 Processor -- start 2024-03-08T08:28:19.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.505+0000 7ff943119640 1 -- start start 2024-03-08T08:28:19.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.505+0000 7ff943119640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff93c15dfa0 0x7ff93c06c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.505+0000 7ff943119640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff93c06b040 0x7ff93c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.505+0000 7ff943119640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c0ad740 0x7ff93c0adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.505+0000 7ff943119640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c15e380 con 0x7ff93c06b040 2024-03-08T08:28:19.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.505+0000 7ff943119640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c0b5f60 con 0x7ff93c15dfa0 2024-03-08T08:28:19.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.505+0000 7ff943119640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0b6240 con 0x7ff93c0ad740 2024-03-08T08:28:19.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff93bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff93c06b040 0x7ff93c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff93bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff93c06b040 0x7ff93c0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42432/0 (socket says 172.21.15.105:42432) 2024-03-08T08:28:19.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff93bfff640 1 -- 172.21.15.105:0/2879965454 learned_addr learned my addr 172.21.15.105:0/2879965454 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:19.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff9416b6640 1 --2- 172.21.15.105:0/2879965454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c0ad740 0x7ff93c0adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff940eb5640 1 --2- 172.21.15.105:0/2879965454 >> v2:172.21.15.144:3300/0 conn(0x7ff93c15dfa0 0x7ff93c06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff93bfff640 1 -- 172.21.15.105:0/2879965454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c0ad740 msgr2=0x7ff93c0adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:19.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff93bfff640 1 --2- 172.21.15.105:0/2879965454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c0ad740 0x7ff93c0adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff93bfff640 1 -- 172.21.15.105:0/2879965454 >> v2:172.21.15.144:3300/0 conn(0x7ff93c15dfa0 msgr2=0x7ff93c06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:19.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff93bfff640 1 --2- 172.21.15.105:0/2879965454 >> v2:172.21.15.144:3300/0 conn(0x7ff93c15dfa0 0x7ff93c06c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.506+0000 7ff93bfff640 1 -- 172.21.15.105:0/2879965454 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c06b720 con 0x7ff93c06b040 2024-03-08T08:28:19.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff93bfff640 1 --2- 172.21.15.105:0/2879965454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c06b040 0x7ff93c0ad200 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff93c073360 tx=0x7ff92402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7ed61bae3d07a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff93b7fe640 1 -- 172.21.15.105:0/2879965454 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924035020 con 0x7ff93c06b040 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff93b7fe640 1 -- 172.21.15.105:0/2879965454 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff924031870 con 0x7ff93c06b040 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff93b7fe640 1 -- 172.21.15.105:0/2879965454 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924031c40 con 0x7ff93c06b040 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 -- 172.21.15.105:0/2879965454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c06b040 msgr2=0x7ff93c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 --2- 172.21.15.105:0/2879965454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c06b040 0x7ff93c0ad200 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff93c073360 tx=0x7ff92402b910 comp rx=0 tx=0).stop 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 -- 172.21.15.105:0/2879965454 shutdown_connections 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 --2- 172.21.15.105:0/2879965454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c0ad740 0x7ff93c0adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 --2- 172.21.15.105:0/2879965454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c06b040 0x7ff93c0ad200 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 --2- 172.21.15.105:0/2879965454 >> v2:172.21.15.144:3300/0 conn(0x7ff93c15dfa0 0x7ff93c06c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 -- 172.21.15.105:0/2879965454 >> 172.21.15.105:0/2879965454 conn(0x7ff93c0b50f0 msgr2=0x7ff93c0b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 -- 172.21.15.105:0/2879965454 shutdown_connections 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.507+0000 7ff943119640 1 -- 172.21.15.105:0/2879965454 wait complete. 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.509+0000 7ff943119640 1 Processor -- start 2024-03-08T08:28:19.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.509+0000 7ff943119640 1 -- start start 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.509+0000 7ff943119640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c06b040 0x7ff93c149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.509+0000 7ff943119640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93c0ad740 0x7ff93c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.509+0000 7ff943119640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c15dfa0 0x7ff93c0d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.509+0000 7ff943119640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c06d000 con 0x7ff93c15dfa0 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.509+0000 7ff943119640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c06bce0 con 0x7ff93c0ad740 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.509+0000 7ff943119640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c06d180 con 0x7ff93c06b040 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c15dfa0 0x7ff93c0d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c15dfa0 0x7ff93c0d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42444/0 (socket says 172.21.15.105:42444) 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 -- 172.21.15.105:0/3250042951 learned_addr learned my addr 172.21.15.105:0/3250042951 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff940eb5640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c06b040 0x7ff93c149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 -- 172.21.15.105:0/3250042951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c06b040 msgr2=0x7ff93c149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c06b040 0x7ff93c149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff93bfff640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93c0ad740 0x7ff93c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 -- 172.21.15.105:0/3250042951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93c0ad740 msgr2=0x7ff93c0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93c0ad740 0x7ff93c0d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 -- 172.21.15.105:0/3250042951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c1f3d50 con 0x7ff93c15dfa0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9416b6640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c15dfa0 0x7ff93c0d06b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00e830 tx=0x7ff92c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9397fa640 1 -- 172.21.15.105:0/3250042951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c017070 con 0x7ff93c15dfa0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9397fa640 1 -- 172.21.15.105:0/3250042951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c00c940 con 0x7ff93c15dfa0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c1f3fc0 con 0x7ff93c15dfa0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff9397fa640 1 -- 172.21.15.105:0/3250042951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00cd30 con 0x7ff93c15dfa0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.510+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c1f4580 con 0x7ff93c15dfa0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.511+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c075120 con 0x7ff93c15dfa0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.512+0000 7ff9397fa640 1 -- 172.21.15.105:0/3250042951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff92c01a070 con 0x7ff93c15dfa0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.512+0000 7ff9397fa640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff91c06b500 0x7ff91c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.512+0000 7ff940eb5640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff91c06b500 0x7ff91c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.512+0000 7ff9397fa640 1 -- 172.21.15.105:0/3250042951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff92c08a180 con 0x7ff93c15dfa0 2024-03-08T08:28:19.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.512+0000 7ff940eb5640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff91c06b500 0x7ff91c06d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff934007380 tx=0x7ff93400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.516+0000 7ff9397fa640 1 -- 172.21.15.105:0/3250042951 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff92c059000 con 0x7ff93c15dfa0 2024-03-08T08:28:19.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.688+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff93c071ae0 con 0x7ff93c15dfa0 2024-03-08T08:28:19.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.688+0000 7ff9397fa640 1 -- 172.21.15.105:0/3250042951 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff92c058900 con 0x7ff93c15dfa0 2024-03-08T08:28:19.690 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff91c06b500 msgr2=0x7ff91c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff91c06b500 0x7ff91c06d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff934007380 tx=0x7ff93400b040 comp rx=0 tx=0).stop 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c15dfa0 msgr2=0x7ff93c0d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c15dfa0 0x7ff93c0d06b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00e830 tx=0x7ff92c00ecf0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 shutdown_connections 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff91c06b500 0x7ff91c06d9c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff93c15dfa0 0x7ff93c0d06b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93c0ad740 0x7ff93c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 --2- 172.21.15.105:0/3250042951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff93c06b040 0x7ff93c149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.690+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 >> 172.21.15.105:0/3250042951 conn(0x7ff93c0b50f0 msgr2=0x7ff93c15d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.691+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 shutdown_connections 2024-03-08T08:28:19.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.691+0000 7ff943119640 1 -- 172.21.15.105:0/3250042951 wait complete. 2024-03-08T08:28:19.702 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:28:19.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.801+0000 7f9495181640 1 Processor -- start 2024-03-08T08:28:19.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.801+0000 7f9495181640 1 -- start start 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f9495181640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f949006d0b0 0x7f94900b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f9495181640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f949006d670 0x7f949006da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f9495181640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949006dfc0 con 0x7f949015ac50 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f9495181640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9490072e90 con 0x7f949006d670 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f9495181640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949015b370 con 0x7f949006d0b0 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f948f59e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f949006d670 0x7f949006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f948ed9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f949015ac50 0x7f949015be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42422/0 (socket says 172.21.15.105:42422) 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f948ed9d640 1 -- 172.21.15.105:0/4122615894 learned_addr learned my addr 172.21.15.105:0/4122615894 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f948f59e640 1 -- 172.21.15.105:0/4122615894 >> v2:172.21.15.144:3300/0 conn(0x7f949006d0b0 msgr2=0x7f94900b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f948f59e640 1 --2- 172.21.15.105:0/4122615894 >> v2:172.21.15.144:3300/0 conn(0x7f949006d0b0 0x7f94900b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f948f59e640 1 -- 172.21.15.105:0/4122615894 >> v1:172.21.15.105:6789/0 conn(0x7f949015ac50 legacy=0x7f949015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f948f59e640 1 -- 172.21.15.105:0/4122615894 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949015b650 con 0x7f949006d670 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.802+0000 7f948e59c640 1 -- 172.21.15.105:0/4122615894 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876077215 0 0) 0x7f9480001110 con 0x7f949015ac50 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.803+0000 7f948f59e640 1 --2- 172.21.15.105:0/4122615894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d670 0x7f949006da80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94900afb90 tx=0x7f948802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e512d0cad86c8a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.803+0000 7f948e59c640 1 -- 172.21.15.105:0/4122615894 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488039070 con 0x7f949006d670 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.803+0000 7f948e59c640 1 -- 172.21.15.105:0/4122615894 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948802cbf0 con 0x7f949006d670 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.803+0000 7f948e59c640 1 -- 172.21.15.105:0/4122615894 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488034440 con 0x7f949006d670 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.804+0000 7f9495181640 1 -- 172.21.15.105:0/4122615894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d670 msgr2=0x7f949006da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.804+0000 7f9495181640 1 --2- 172.21.15.105:0/4122615894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d670 0x7f949006da80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f94900afb90 tx=0x7f948802be40 comp rx=0 tx=0).stop 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.804+0000 7f9495181640 1 -- 172.21.15.105:0/4122615894 shutdown_connections 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.804+0000 7f9495181640 1 --2- 172.21.15.105:0/4122615894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d670 0x7f949006da80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.804+0000 7f9495181640 1 --2- 172.21.15.105:0/4122615894 >> v2:172.21.15.144:3300/0 conn(0x7f949006d0b0 0x7f94900b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.804+0000 7f9495181640 1 -- 172.21.15.105:0/4122615894 >> 172.21.15.105:0/4122615894 conn(0x7f94900b50f0 msgr2=0x7f94900b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.804+0000 7f9495181640 1 -- 172.21.15.105:0/4122615894 shutdown_connections 2024-03-08T08:28:19.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.804+0000 7f9495181640 1 -- 172.21.15.105:0/4122615894 wait complete. 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.805+0000 7f9495181640 1 Processor -- start 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.805+0000 7f9495181640 1 -- start start 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f9495181640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d0b0 0x7f949006bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f9495181640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f949006d670 0x7f949006c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f9495181640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f949015ac50 0x7f9490147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f9495181640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490073170 con 0x7f949006d0b0 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f9495181640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949015b030 con 0x7f949006d670 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f9495181640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949006ac80 con 0x7f949015ac50 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d0b0 0x7f949006bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d0b0 0x7f949006bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42454/0 (socket says 172.21.15.105:42454) 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 -- 172.21.15.105:0/1251978018 learned_addr learned my addr 172.21.15.105:0/1251978018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f9487fff640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f949006d670 0x7f949006c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948f59e640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f949015ac50 0x7f9490147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 -- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f949015ac50 msgr2=0x7f9490147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f949015ac50 0x7f9490147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 -- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f949006d670 msgr2=0x7f949006c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f949006d670 0x7f949006c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 -- 172.21.15.105:0/1251978018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949014bcb0 con 0x7f949006d0b0 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f9487fff640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f949006d670 0x7f949006c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f948ed9d640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d0b0 0x7f949006bc70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f948000c430 tx=0x7f948000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.806+0000 7f94877fe640 1 -- 172.21.15.105:0/1251978018 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94800074e0 con 0x7f949006d0b0 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.807+0000 7f94877fe640 1 -- 172.21.15.105:0/1251978018 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94800047e0 con 0x7f949006d0b0 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.807+0000 7f94877fe640 1 -- 172.21.15.105:0/1251978018 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480004bd0 con 0x7f949006d0b0 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.807+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9490148460 con 0x7f949006d0b0 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.807+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9490145890 con 0x7f949006d0b0 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.808+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9490071120 con 0x7f949006d0b0 2024-03-08T08:28:19.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.808+0000 7f94877fe640 1 -- 172.21.15.105:0/1251978018 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9480011070 con 0x7f949006d0b0 2024-03-08T08:28:19.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.808+0000 7f94877fe640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f947006b4d0 0x7f947006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:19.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.809+0000 7f9487fff640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f947006b4d0 0x7f947006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:19.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.809+0000 7f94877fe640 1 -- 172.21.15.105:0/1251978018 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9480089e30 con 0x7f949006d0b0 2024-03-08T08:28:19.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.809+0000 7f9487fff640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f947006b4d0 0x7f947006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f94900b2fe0 tx=0x7f9478008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:19.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:19.817+0000 7f94877fe640 1 -- 172.21.15.105:0/1251978018 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9490071120 con 0x7f949006d0b0 2024-03-08T08:28:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.027+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9490075120 con 0x7f949006d0b0 2024-03-08T08:28:20.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.027+0000 7f94877fe640 1 -- 172.21.15.105:0/1251978018 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4944 (secure 0 0 0) 0x7f949006dfc0 con 0x7f949006d0b0 2024-03-08T08:28:20.029 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:20.029 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:27:38.998370+0000","modified":"2024-03-08T08:27:41.001533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T08:28:20.029 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f947006b4d0 msgr2=0x7f947006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f947006b4d0 0x7f947006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f94900b2fe0 tx=0x7f9478008040 comp rx=0 tx=0).stop 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d0b0 msgr2=0x7f949006bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d0b0 0x7f949006bc70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f948000c430 tx=0x7f948000c8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 shutdown_connections 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f947006b4d0 0x7f947006d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f949015ac50 0x7f9490147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f949006d670 0x7f949006c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 --2- 172.21.15.105:0/1251978018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f949006d0b0 0x7f949006bc70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.029+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 >> 172.21.15.105:0/1251978018 conn(0x7f94900b50f0 msgr2=0x7f9490156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.030+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 shutdown_connections 2024-03-08T08:28:20.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.030+0000 7f9495181640 1 -- 172.21.15.105:0/1251978018 wait complete. 2024-03-08T08:28:20.041 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.140+0000 7f52925b5640 1 Processor -- start 2024-03-08T08:28:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.141+0000 7f52925b5640 1 -- start start 2024-03-08T08:28:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.141+0000 7f52925b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.141+0000 7f52925b5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f528c06ccc0 0x7f528c0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.141+0000 7f52925b5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c0b2760 con 0x7f528c15dfc0 2024-03-08T08:28:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.141+0000 7f52925b5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c0b3370 con 0x7f528c06ccc0 2024-03-08T08:28:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.141+0000 7f52925b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0b2a40 con 0x7f528c0afc30 2024-03-08T08:28:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f5290b52640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f528c06ccc0 0x7f528c0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f528bfff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f528c15dfc0 0x7f528c0b1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:35686/0 (socket says 172.21.15.105:35686) 2024-03-08T08:28:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c06c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:41400/0 (socket says 172.21.15.105:41400) 2024-03-08T08:28:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f528bfff640 1 -- 172.21.15.105:0/1254937183 learned_addr learned my addr 172.21.15.105:0/1254937183 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f5290b52640 1 -- 172.21.15.105:0/1254937183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 msgr2=0x7f528c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f5290b52640 1 --2- 172.21.15.105:0/1254937183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c06c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f5290b52640 1 -- 172.21.15.105:0/1254937183 >> v1:172.21.15.144:6789/0 conn(0x7f528c15dfc0 legacy=0x7f528c0b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f5290b52640 1 -- 172.21.15.105:0/1254937183 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0b2d20 con 0x7f528c06ccc0 2024-03-08T08:28:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.142+0000 7f528b7fe640 1 --2- 172.21.15.105:0/1254937183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:20.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.143+0000 7f5290b52640 1 --2- 172.21.15.105:0/1254937183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 0x7f528c0b54a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5284009d90 tx=0x7f528402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3038195d2532a0a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.143+0000 7f528affd640 1 -- 172.21.15.105:0/1254937183 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284039070 con 0x7f528c06ccc0 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.143+0000 7f528affd640 1 -- 172.21.15.105:0/1254937183 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528402cc50 con 0x7f528c06ccc0 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.143+0000 7f528affd640 1 -- 172.21.15.105:0/1254937183 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284034440 con 0x7f528c06ccc0 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.143+0000 7f52925b5640 1 -- 172.21.15.105:0/1254937183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 msgr2=0x7f528c0b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.143+0000 7f52925b5640 1 --2- 172.21.15.105:0/1254937183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 0x7f528c0b54a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5284009d90 tx=0x7f528402bea0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.143+0000 7f52925b5640 1 -- 172.21.15.105:0/1254937183 shutdown_connections 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.143+0000 7f52925b5640 1 --2- 172.21.15.105:0/1254937183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 0x7f528c0b54a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.144+0000 7f52925b5640 1 --2- 172.21.15.105:0/1254937183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.144+0000 7f52925b5640 1 -- 172.21.15.105:0/1254937183 >> 172.21.15.105:0/1254937183 conn(0x7f528c09aa00 msgr2=0x7f528c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.144+0000 7f52925b5640 1 -- 172.21.15.105:0/1254937183 shutdown_connections 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.144+0000 7f52925b5640 1 -- 172.21.15.105:0/1254937183 wait complete. 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.145+0000 7f52925b5640 1 Processor -- start 2024-03-08T08:28:20.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.145+0000 7f52925b5640 1 -- start start 2024-03-08T08:28:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.145+0000 7f52925b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 0x7f528c1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.145+0000 7f52925b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c1504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.145+0000 7f52925b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c15dfc0 0x7f528c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.145+0000 7f52925b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c15e520 con 0x7f528c15dfc0 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.145+0000 7f52925b5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c15e3a0 con 0x7f528c06ccc0 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.145+0000 7f52925b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0b59e0 con 0x7f528c0afc30 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c15dfc0 0x7f528c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c15dfc0 0x7f528c0aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42470/0 (socket says 172.21.15.105:42470) 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 -- 172.21.15.105:0/1443910428 learned_addr learned my addr 172.21.15.105:0/1443910428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f528b7fe640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c1504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f528bfff640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 0x7f528c1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 -- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 msgr2=0x7f528c1504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c1504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 -- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 msgr2=0x7f528c1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 0x7f528c1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 -- 172.21.15.105:0/1443910428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0d2570 con 0x7f528c15dfc0 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f528bfff640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 0x7f528c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5290b52640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c15dfc0 0x7f528c0aed90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5284037040 tx=0x7f5284004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5288ff9640 1 -- 172.21.15.105:0/1443910428 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284044020 con 0x7f528c15dfc0 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5288ff9640 1 -- 172.21.15.105:0/1443910428 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528402ca50 con 0x7f528c15dfc0 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c15e6a0 con 0x7f528c15dfc0 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f5288ff9640 1 -- 172.21.15.105:0/1443910428 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284039040 con 0x7f528c15dfc0 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.146+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c15ec60 con 0x7f528c15dfc0 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.147+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c15f700 con 0x7f528c15dfc0 2024-03-08T08:28:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.148+0000 7f5288ff9640 1 -- 172.21.15.105:0/1443910428 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5284047020 con 0x7f528c15dfc0 2024-03-08T08:28:20.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.148+0000 7f5288ff9640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f527806b4d0 0x7f527806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.148+0000 7f5288ff9640 1 -- 172.21.15.105:0/1443910428 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f52840acb40 con 0x7f528c15dfc0 2024-03-08T08:28:20.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.148+0000 7f528bfff640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f527806b4d0 0x7f527806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.149+0000 7f528bfff640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f527806b4d0 0x7f527806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f527c000f60 tx=0x7f527c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.153+0000 7f5288ff9640 1 -- 172.21.15.105:0/1443910428 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f528c15f700 con 0x7f528c15dfc0 2024-03-08T08:28:20.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.324+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f528c075120 con 0x7f528c15dfc0 2024-03-08T08:28:20.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.325+0000 7f5288ff9640 1 -- 172.21.15.105:0/1443910428 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f528404e030 con 0x7f528c15dfc0 2024-03-08T08:28:20.326 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:20.326 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":21,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:19.803815+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f527806b4d0 msgr2=0x7f527806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f527806b4d0 0x7f527806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f527c000f60 tx=0x7f527c009040 comp rx=0 tx=0).stop 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c15dfc0 msgr2=0x7f528c0aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c15dfc0 0x7f528c0aed90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5284037040 tx=0x7f5284004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 shutdown_connections 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f527806b4d0 0x7f527806d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f528c15dfc0 0x7f528c0aed90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f528c0afc30 0x7f528c1504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 --2- 172.21.15.105:0/1443910428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f528c06ccc0 0x7f528c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 >> 172.21.15.105:0/1443910428 conn(0x7f528c09aa00 msgr2=0x7f528c06f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:20.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 shutdown_connections 2024-03-08T08:28:20.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.327+0000 7f52925b5640 1 -- 172.21.15.105:0/1443910428 wait complete. 2024-03-08T08:28:20.339 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T08:28:20.339 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:28:20.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.437+0000 7fe2bf7c7640 1 Processor -- start 2024-03-08T08:28:20.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.437+0000 7fe2bf7c7640 1 -- start start 2024-03-08T08:28:20.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bf7c7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2b815dfa0 0x7fe2b806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bf7c7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bf7c7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b815e380 con 0x7fe2b80ad740 2024-03-08T08:28:20.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bf7c7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b80b5f60 con 0x7fe2b815dfa0 2024-03-08T08:28:20.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bf7c7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b6240 con 0x7fe2b806b040 2024-03-08T08:28:20.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bd563640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2b815dfa0 0x7fe2b806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bdd64640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe2b80ad740 0x7fe2b80d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42432/0 (socket says 172.21.15.105:42432) 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bdd64640 1 -- 172.21.15.105:0/1498007248 learned_addr learned my addr 172.21.15.105:0/1498007248 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bcd62640 1 --2- 172.21.15.105:0/1498007248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bd563640 1 -- 172.21.15.105:0/1498007248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 msgr2=0x7fe2b80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bd563640 1 --2- 172.21.15.105:0/1498007248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b80ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bd563640 1 -- 172.21.15.105:0/1498007248 >> v1:172.21.15.105:6789/0 conn(0x7fe2b80ad740 legacy=0x7fe2b80d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bd563640 1 -- 172.21.15.105:0/1498007248 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b806b720 con 0x7fe2b815dfa0 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2affff640 1 -- 172.21.15.105:0/1498007248 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942350123 0 0) 0x7fe2b815e380 con 0x7fe2b80ad740 2024-03-08T08:28:20.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.438+0000 7fe2bcd62640 1 --2- 172.21.15.105:0/1498007248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:20.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2bd563640 1 --2- 172.21.15.105:0/1498007248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b815dfa0 0x7fe2b806c200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40097a0 tx=0x7fe2b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fcaf3aa0b849c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2affff640 1 -- 172.21.15.105:0/1498007248 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4035020 con 0x7fe2b815dfa0 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2affff640 1 -- 172.21.15.105:0/1498007248 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b4031830 con 0x7fe2b815dfa0 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2affff640 1 -- 172.21.15.105:0/1498007248 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4031c00 con 0x7fe2b815dfa0 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/1498007248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b815dfa0 msgr2=0x7fe2b806c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/1498007248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b815dfa0 0x7fe2b806c200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40097a0 tx=0x7fe2b402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/1498007248 shutdown_connections 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/1498007248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.439+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/1498007248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b815dfa0 0x7fe2b806c200 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.440+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/1498007248 >> 172.21.15.105:0/1498007248 conn(0x7fe2b80b50f0 msgr2=0x7fe2b80b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.440+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/1498007248 shutdown_connections 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.440+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/1498007248 wait complete. 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bf7c7640 1 Processor -- start 2024-03-08T08:28:20.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bf7c7640 1 -- start start 2024-03-08T08:28:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bf7c7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b81495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bf7c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b80ad740 0x7fe2b8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bf7c7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b815dfa0 0x7fe2b8153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bf7c7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806d000 con 0x7fe2b80ad740 2024-03-08T08:28:20.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bf7c7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806bce0 con 0x7fe2b815dfa0 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bf7c7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b806d180 con 0x7fe2b806b040 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.441+0000 7fe2bcd62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b80ad740 0x7fe2b8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bcd62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b80ad740 0x7fe2b8156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42486/0 (socket says 172.21.15.105:42486) 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bcd62640 1 -- 172.21.15.105:0/3644409751 learned_addr learned my addr 172.21.15.105:0/3644409751 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bcd62640 1 -- 172.21.15.105:0/3644409751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 msgr2=0x7fe2b81495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bd563640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b81495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bcd62640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b81495f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bcd62640 1 -- 172.21.15.105:0/3644409751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b815dfa0 msgr2=0x7fe2b8153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bdd64640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b815dfa0 0x7fe2b8153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bcd62640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b815dfa0 0x7fe2b8153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bcd62640 1 -- 172.21.15.105:0/3644409751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b8149b30 con 0x7fe2b80ad740 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bcd62640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b80ad740 0x7fe2b8156630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c7f0 tx=0x7fe2a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2adffb640 1 -- 172.21.15.105:0/3644409751 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8015070 con 0x7fe2b80ad740 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2adffb640 1 -- 172.21.15.105:0/3644409751 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8007a30 con 0x7fe2b80ad740 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b8157080 con 0x7fe2b80ad740 2024-03-08T08:28:20.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2adffb640 1 -- 172.21.15.105:0/3644409751 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8007e00 con 0x7fe2b80ad740 2024-03-08T08:28:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.442+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b8157920 con 0x7fe2b80ad740 2024-03-08T08:28:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.443+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b8075120 con 0x7fe2b80ad740 2024-03-08T08:28:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.444+0000 7fe2adffb640 1 -- 172.21.15.105:0/3644409751 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2a8018070 con 0x7fe2b80ad740 2024-03-08T08:28:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.444+0000 7fe2adffb640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe29c06b4d0 0x7fe29c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.444+0000 7fe2adffb640 1 -- 172.21.15.105:0/3644409751 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe2a808a1f0 con 0x7fe2b80ad740 2024-03-08T08:28:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.444+0000 7fe2bd563640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe29c06b4d0 0x7fe29c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.445+0000 7fe2bd563640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe29c06b4d0 0x7fe29c06d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4036000 tx=0x7fe2b403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.448+0000 7fe2adffb640 1 -- 172.21.15.105:0/3644409751 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2b8149b30 con 0x7fe2b80ad740 2024-03-08T08:28:20.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.654+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2b8071ae0 con 0x7fe2b80ad740 2024-03-08T08:28:20.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.654+0000 7fe2adffb640 1 -- 172.21.15.105:0/3644409751 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4944 (secure 0 0 0) 0x7fe2a8058ff0 con 0x7fe2b80ad740 2024-03-08T08:28:20.656 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:20.656 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:27:38.998370+0000","modified":"2024-03-08T08:27:41.001533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T08:28:20.656 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.656+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe29c06b4d0 msgr2=0x7fe29c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.656+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe29c06b4d0 0x7fe29c06d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4036000 tx=0x7fe2b403c040 comp rx=0 tx=0).stop 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.656+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b80ad740 msgr2=0x7fe2b8156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.656+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b80ad740 0x7fe2b8156630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c7f0 tx=0x7fe2a800cbf0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.657+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 shutdown_connections 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.657+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe29c06b4d0 0x7fe29c06d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.657+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b815dfa0 0x7fe2b8153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.657+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b80ad740 0x7fe2b8156630 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.657+0000 7fe2bf7c7640 1 --2- 172.21.15.105:0/3644409751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2b806b040 0x7fe2b81495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.657+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 >> 172.21.15.105:0/3644409751 conn(0x7fe2b80b50f0 msgr2=0x7fe2b815cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:20.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.657+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 shutdown_connections 2024-03-08T08:28:20.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.657+0000 7fe2bf7c7640 1 -- 172.21.15.105:0/3644409751 wait complete. 2024-03-08T08:28:20.669 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:28:20.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.767+0000 7f6995c5c640 1 Processor -- start 2024-03-08T08:28:20.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.767+0000 7f6995c5c640 1 -- start start 2024-03-08T08:28:20.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f6995c5c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699015ac50 0x7f69900b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f6995c5c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f699006d670 0x7f699006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f6995c5c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6990072e90 con 0x7f699006d0b0 2024-03-08T08:28:20.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f6995c5c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699006e000 con 0x7f699006d670 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f6995c5c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699015b370 con 0x7f699015ac50 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f699006d670 0x7f699006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698effd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f699006d0b0 0x7f699015be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42444/0 (socket says 172.21.15.105:42444) 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698effd640 1 -- 172.21.15.105:0/289863322 learned_addr learned my addr 172.21.15.105:0/289863322 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698f7fe640 1 --2- 172.21.15.105:0/289863322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699015ac50 0x7f69900b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698ffff640 1 -- 172.21.15.105:0/289863322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699015ac50 msgr2=0x7f69900b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698e7fc640 1 -- 172.21.15.105:0/289863322 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258125308 0 0) 0x7f6990072e90 con 0x7f699006d0b0 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698ffff640 1 --2- 172.21.15.105:0/289863322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699015ac50 0x7f69900b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698ffff640 1 -- 172.21.15.105:0/289863322 >> v1:172.21.15.105:6789/0 conn(0x7f699006d0b0 legacy=0x7f699015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698ffff640 1 -- 172.21.15.105:0/289863322 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699015b650 con 0x7f699006d670 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.768+0000 7f698f7fe640 1 --2- 172.21.15.105:0/289863322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699015ac50 0x7f69900b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.769+0000 7f698ffff640 1 --2- 172.21.15.105:0/289863322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699006d670 0x7f699006dac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6988009d90 tx=0x7f698802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abcfc514fc0bc44 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.769+0000 7f698e7fc640 1 -- 172.21.15.105:0/289863322 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988039070 con 0x7f699006d670 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.769+0000 7f698e7fc640 1 -- 172.21.15.105:0/289863322 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698802cbc0 con 0x7f699006d670 2024-03-08T08:28:20.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.769+0000 7f698e7fc640 1 -- 172.21.15.105:0/289863322 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988034440 con 0x7f699006d670 2024-03-08T08:28:20.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.770+0000 7f6995c5c640 1 -- 172.21.15.105:0/289863322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699006d670 msgr2=0x7f699006dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.770+0000 7f6995c5c640 1 --2- 172.21.15.105:0/289863322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699006d670 0x7f699006dac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6988009d90 tx=0x7f698802be10 comp rx=0 tx=0).stop 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.770+0000 7f6995c5c640 1 -- 172.21.15.105:0/289863322 shutdown_connections 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.770+0000 7f6995c5c640 1 --2- 172.21.15.105:0/289863322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699006d670 0x7f699006dac0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.770+0000 7f6995c5c640 1 --2- 172.21.15.105:0/289863322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699015ac50 0x7f69900b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.770+0000 7f6995c5c640 1 -- 172.21.15.105:0/289863322 >> 172.21.15.105:0/289863322 conn(0x7f69900b50f0 msgr2=0x7f69900b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.770+0000 7f6995c5c640 1 -- 172.21.15.105:0/289863322 shutdown_connections 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.770+0000 7f6995c5c640 1 -- 172.21.15.105:0/289863322 wait complete. 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.771+0000 7f6995c5c640 1 Processor -- start 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.771+0000 7f6995c5c640 1 -- start start 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f6995c5c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699006d0b0 0x7f699009d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f6995c5c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699006d670 0x7f6990148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f6995c5c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015ac50 0x7f6990149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f6995c5c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6990073170 con 0x7f699015ac50 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f6995c5c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699015b030 con 0x7f699006d0b0 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f6995c5c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699006ac80 con 0x7f699006d670 2024-03-08T08:28:20.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015ac50 0x7f6990149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015ac50 0x7f6990149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42512/0 (socket says 172.21.15.105:42512) 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 -- 172.21.15.105:0/25522344 learned_addr learned my addr 172.21.15.105:0/25522344 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698effd640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699006d670 0x7f6990148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698f7fe640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699006d0b0 0x7f699009d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 -- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699006d670 msgr2=0x7f6990148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699006d670 0x7f6990148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 -- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699006d0b0 msgr2=0x7f699009d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699006d0b0 0x7f699009d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 -- 172.21.15.105:0/25522344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6990149bf0 con 0x7f699015ac50 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698f7fe640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699006d0b0 0x7f699009d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.772+0000 7f698ffff640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015ac50 0x7f6990149240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6988009d90 tx=0x7f6988004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.773+0000 7f6977fff640 1 -- 172.21.15.105:0/25522344 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988039050 con 0x7f699015ac50 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.773+0000 7f6977fff640 1 -- 172.21.15.105:0/25522344 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6988045070 con 0x7f699015ac50 2024-03-08T08:28:20.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.773+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6990149ed0 con 0x7f699015ac50 2024-03-08T08:28:20.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.773+0000 7f6977fff640 1 -- 172.21.15.105:0/25522344 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698803e830 con 0x7f699015ac50 2024-03-08T08:28:20.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.773+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699014a490 con 0x7f699015ac50 2024-03-08T08:28:20.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.774+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69900b6580 con 0x7f699015ac50 2024-03-08T08:28:20.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.774+0000 7f6977fff640 1 -- 172.21.15.105:0/25522344 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f698803e9d0 con 0x7f699015ac50 2024-03-08T08:28:20.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.774+0000 7f6977fff640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f697c06b4d0 0x7f697c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:20.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.775+0000 7f6977fff640 1 -- 172.21.15.105:0/25522344 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f69880ad2a0 con 0x7f699015ac50 2024-03-08T08:28:20.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.775+0000 7f698f7fe640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f697c06b4d0 0x7f697c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:20.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.775+0000 7f698f7fe640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f697c06b4d0 0x7f697c06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6980006fd0 tx=0x7f6980008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:20.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.779+0000 7f6977fff640 1 -- 172.21.15.105:0/25522344 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69900b6580 con 0x7f699015ac50 2024-03-08T08:28:20.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.990+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6990075120 con 0x7f699015ac50 2024-03-08T08:28:20.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.990+0000 7f6977fff640 1 -- 172.21.15.105:0/25522344 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4944 (secure 0 0 0) 0x7f698807c1c0 con 0x7f699015ac50 2024-03-08T08:28:20.992 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:20.992 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/3967292194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3967292194},{"type":"v1","addr":"172.21.15.144:6834","nonce":3967292194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2713558266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2713558266},{"type":"v1","addr":"172.21.15.105:6835","nonce":2713558266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/1809867135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":1809867135},{"type":"v1","addr":"172.21.15.105:6837","nonce":1809867135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:27:38.998370+0000","modified":"2024-03-08T08:27:41.001533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/3724471877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3724471877},{"type":"v1","addr":"172.21.15.144:6835","nonce":3724471877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T08:28:20.992 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f697c06b4d0 msgr2=0x7f697c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f697c06b4d0 0x7f697c06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6980006fd0 tx=0x7f6980008040 comp rx=0 tx=0).stop 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015ac50 msgr2=0x7f6990149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015ac50 0x7f6990149240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6988009d90 tx=0x7f6988004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 shutdown_connections 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f697c06b4d0 0x7f697c06d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015ac50 0x7f6990149240 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699006d670 0x7f6990148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 --2- 172.21.15.105:0/25522344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699006d0b0 0x7f699009d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:20.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.992+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 >> 172.21.15.105:0/25522344 conn(0x7f69900b50f0 msgr2=0x7f69900ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:20.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.993+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 shutdown_connections 2024-03-08T08:28:20.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:20.993+0000 7f6995c5c640 1 -- 172.21.15.105:0/25522344 wait complete. 2024-03-08T08:28:21.004 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.105+0000 7f5eb092c640 1 Processor -- start 2024-03-08T08:28:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.105+0000 7f5eb092c640 1 -- start start 2024-03-08T08:28:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.106+0000 7f5eb092c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5eac0b4e40 0x7f5eac0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.106+0000 7f5eb092c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5eac0b5a60 0x7f5eac15c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.106+0000 7f5eb092c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac15f7b0 con 0x7f5eac15bea0 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.106+0000 7f5eb092c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac15e710 con 0x7f5eac0b5a60 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.106+0000 7f5eb092c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac15e9f0 con 0x7f5eac0b4e40 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.106+0000 7f5eaad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5eac0b5a60 0x7f5eac15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.107+0000 7f5eaad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5eac0b5a60 0x7f5eac15c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42524/0 (socket says 172.21.15.105:42524) 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.107+0000 7f5eaad9d640 1 -- 172.21.15.105:0/872448526 learned_addr learned my addr 172.21.15.105:0/872448526 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.107+0000 7f5ea9d9b640 1 --2- 172.21.15.105:0/872448526 >> v2:172.21.15.144:3300/0 conn(0x7f5eac0b4e40 0x7f5eac0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.107+0000 7f5eaad9d640 1 -- 172.21.15.105:0/872448526 >> v2:172.21.15.144:3300/0 conn(0x7f5eac0b4e40 msgr2=0x7f5eac0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.107+0000 7f5eaad9d640 1 --2- 172.21.15.105:0/872448526 >> v2:172.21.15.144:3300/0 conn(0x7f5eac0b4e40 0x7f5eac0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.107+0000 7f5eaad9d640 1 -- 172.21.15.105:0/872448526 >> v1:172.21.15.144:6789/0 conn(0x7f5eac15bea0 legacy=0x7f5eac15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.107+0000 7f5eaad9d640 1 -- 172.21.15.105:0/872448526 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac15ecd0 con 0x7f5eac0b5a60 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.107+0000 7f5eaad9d640 1 --2- 172.21.15.105:0/872448526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b5a60 0x7f5eac15c280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5eac06bad0 tx=0x7f5e9c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8947876548f82a8d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5ea959a640 1 -- 172.21.15.105:0/872448526 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c02ca80 con 0x7f5eac0b5a60 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5ea959a640 1 -- 172.21.15.105:0/872448526 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c02cc20 con 0x7f5eac0b5a60 2024-03-08T08:28:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5ea959a640 1 -- 172.21.15.105:0/872448526 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0343f0 con 0x7f5eac0b5a60 2024-03-08T08:28:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5eb092c640 1 -- 172.21.15.105:0/872448526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b5a60 msgr2=0x7f5eac15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5eb092c640 1 --2- 172.21.15.105:0/872448526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b5a60 0x7f5eac15c280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5eac06bad0 tx=0x7f5e9c02bee0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5eb092c640 1 -- 172.21.15.105:0/872448526 shutdown_connections 2024-03-08T08:28:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5eb092c640 1 --2- 172.21.15.105:0/872448526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b5a60 0x7f5eac15c280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5eb092c640 1 --2- 172.21.15.105:0/872448526 >> v2:172.21.15.144:3300/0 conn(0x7f5eac0b4e40 0x7f5eac0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5eb092c640 1 -- 172.21.15.105:0/872448526 >> 172.21.15.105:0/872448526 conn(0x7f5eac09aa00 msgr2=0x7f5eac09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5eb092c640 1 -- 172.21.15.105:0/872448526 shutdown_connections 2024-03-08T08:28:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.108+0000 7f5eb092c640 1 -- 172.21.15.105:0/872448526 wait complete. 2024-03-08T08:28:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eb092c640 1 Processor -- start 2024-03-08T08:28:21.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eb092c640 1 -- start start 2024-03-08T08:28:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eb092c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b4e40 0x7f5eac0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eb092c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eac0b5a60 0x7f5eac0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eb092c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eac15bea0 0x7f5eac15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eb092c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac09cb00 con 0x7f5eac0b4e40 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eb092c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac15f290 con 0x7f5eac15bea0 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eb092c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac15c7c0 con 0x7f5eac0b5a60 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.110+0000 7f5eaa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b4e40 0x7f5eac0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b4e40 0x7f5eac0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42530/0 (socket says 172.21.15.105:42530) 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaa59c640 1 -- 172.21.15.105:0/162352297 learned_addr learned my addr 172.21.15.105:0/162352297 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaad9d640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eac15bea0 0x7f5eac15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5ea9d9b640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eac0b5a60 0x7f5eac0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaa59c640 1 -- 172.21.15.105:0/162352297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eac0b5a60 msgr2=0x7f5eac0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaa59c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eac0b5a60 0x7f5eac0cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaa59c640 1 -- 172.21.15.105:0/162352297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eac15bea0 msgr2=0x7f5eac15ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaa59c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eac15bea0 0x7f5eac15ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaa59c640 1 -- 172.21.15.105:0/162352297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac0cc520 con 0x7f5eac0b4e40 2024-03-08T08:28:21.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eaa59c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b4e40 0x7f5eac0b2b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400b3f0 tx=0x7f5ea400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5e9affd640 1 -- 172.21.15.105:0/162352297 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4015020 con 0x7f5eac0b4e40 2024-03-08T08:28:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5e9affd640 1 -- 172.21.15.105:0/162352297 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea4011830 con 0x7f5eac0b4e40 2024-03-08T08:28:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5e9affd640 1 -- 172.21.15.105:0/162352297 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4011c20 con 0x7f5eac0b4e40 2024-03-08T08:28:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac1f70b0 con 0x7f5eac0b4e40 2024-03-08T08:28:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.111+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac1f7b90 con 0x7f5eac0b4e40 2024-03-08T08:28:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.112+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac075120 con 0x7f5eac0b4e40 2024-03-08T08:28:21.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.113+0000 7f5e9affd640 1 -- 172.21.15.105:0/162352297 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ea4018020 con 0x7f5eac0b4e40 2024-03-08T08:28:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.113+0000 7f5e9affd640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5e8c06b4d0 0x7f5e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.113+0000 7f5e9affd640 1 -- 172.21.15.105:0/162352297 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5ea4089e20 con 0x7f5eac0b4e40 2024-03-08T08:28:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.113+0000 7f5ea9d9b640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5e8c06b4d0 0x7f5e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.113+0000 7f5ea9d9b640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5e8c06b4d0 0x7f5e8c06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0703a0 tx=0x7f5ea0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.118+0000 7f5e9affd640 1 -- 172.21.15.105:0/162352297 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ea401f030 con 0x7f5eac0b4e40 2024-03-08T08:28:21.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.289+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eac15e010 con 0x7f5eac0b4e40 2024-03-08T08:28:21.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.290+0000 7f5e9affd640 1 -- 172.21.15.105:0/162352297 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f5ea40172c0 con 0x7f5eac0b4e40 2024-03-08T08:28:21.292 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:21.292 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":22,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:20.808708+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:27:37.021001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T08:27:37.970196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5e8c06b4d0 msgr2=0x7f5e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5e8c06b4d0 0x7f5e8c06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0703a0 tx=0x7f5ea0008040 comp rx=0 tx=0).stop 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b4e40 msgr2=0x7f5eac0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b4e40 0x7f5eac0b2b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400b3f0 tx=0x7f5ea400b8b0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 shutdown_connections 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5e8c06b4d0 0x7f5e8c06d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eac15bea0 0x7f5eac15ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5eac0b5a60 0x7f5eac0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 --2- 172.21.15.105:0/162352297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eac0b4e40 0x7f5eac0b2b90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 >> 172.21.15.105:0/162352297 conn(0x7f5eac09aa00 msgr2=0x7f5eac15cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 shutdown_connections 2024-03-08T08:28:21.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.292+0000 7f5eb092c640 1 -- 172.21.15.105:0/162352297 wait complete. 2024-03-08T08:28:21.305 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-08T08:28:21.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.405+0000 7f62de7ac640 1 Processor -- start 2024-03-08T08:28:21.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.405+0000 7f62de7ac640 1 -- start start 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62de7ac640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f62d80d06e0 0x7f62d815f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62de7ac640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d80b2740 0x7f62d80b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62de7ac640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d80b2c20 con 0x7f62d80cfdb0 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62de7ac640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80af4c0 con 0x7f62d80d06e0 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62de7ac640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80af7a0 con 0x7f62d80b2740 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f62d80cfdb0 0x7f62d815ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42454/0 (socket says 172.21.15.105:42454) 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d7fff640 1 -- 172.21.15.105:0/4224112853 learned_addr learned my addr 172.21.15.105:0/4224112853 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62dcd49640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d80b2740 0x7f62d80b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746286755 0 0) 0x7f62d80b2c20 con 0x7f62d80cfdb0 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c0003520 con 0x7f62d80cfdb0 2024-03-08T08:28:21.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d77fe640 1 --2- 172.21.15.105:0/4224112853 >> v2:172.21.15.144:3300/0 conn(0x7f62d80d06e0 0x7f62d815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 489207563 0 0) 0x7f62c0003520 con 0x7f62d80cfdb0 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d80b2740 msgr2=0x7f62d80b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d80b2740 0x7f62d80b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 >> v2:172.21.15.144:3300/0 conn(0x7f62d80d06e0 msgr2=0x7f62d815f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 --2- 172.21.15.105:0/4224112853 >> v2:172.21.15.144:3300/0 conn(0x7f62d80d06e0 0x7f62d815f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80cc490 con 0x7f62d80cfdb0 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f62c8001670 con 0x7f62d80cfdb0 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d77fe640 1 --2- 172.21.15.105:0/4224112853 >> v2:172.21.15.144:3300/0 conn(0x7f62d80d06e0 0x7f62d815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 >> v1:172.21.15.105:6789/0 conn(0x7f62d80cfdb0 legacy=0x7f62d815ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62c00039b0 0x7f62c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d80b2740 0x7f62c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c0004d40 0x7f62c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80af4c0 con 0x7f62c0004d40 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80af7a0 con 0x7f62c00039b0 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c000ba70 con 0x7f62d80b2740 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.406+0000 7f62dcd49640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c0004d40 0x7f62c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.407+0000 7f62d7fff640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62c00039b0 0x7f62c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.407+0000 7f62d77fe640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d80b2740 0x7f62c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.407+0000 7f62dcd49640 1 -- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d80b2740 msgr2=0x7f62c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.407+0000 7f62dcd49640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d80b2740 0x7f62c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.407+0000 7f62dcd49640 1 -- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62c00039b0 msgr2=0x7f62c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.407+0000 7f62dcd49640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62c00039b0 0x7f62c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.407+0000 7f62dcd49640 1 -- 172.21.15.105:0/4224112853 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c0003520 con 0x7f62c0004d40 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.407+0000 7f62dcd49640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c0004d40 0x7f62c00051c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009f90 tx=0x7f62c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a20d0f078692c20 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0039070 con 0x7f62c0004d40 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d002cbd0 con 0x7f62c0004d40 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62d6ffd640 1 -- 172.21.15.105:0/4224112853 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0009c10 con 0x7f62c0004d40 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62de7ac640 1 -- 172.21.15.105:0/4224112853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c0004d40 msgr2=0x7f62c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62de7ac640 1 --2- 172.21.15.105:0/4224112853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62c0004d40 0x7f62c00051c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009f90 tx=0x7f62c0005830 comp rx=0 tx=0).stop 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62dcd49640 1 -- 172.21.15.105:0/4224112853 reap_dead start 2024-03-08T08:28:21.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62de7ac640 1 -- 172.21.15.105:0/4224112853 shutdown_connections 2024-03-08T08:28:21.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62de7ac640 1 -- 172.21.15.105:0/4224112853 >> 172.21.15.105:0/4224112853 conn(0x7f62d80b36a0 msgr2=0x7f62d80b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:21.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62de7ac640 1 -- 172.21.15.105:0/4224112853 shutdown_connections 2024-03-08T08:28:21.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.408+0000 7f62de7ac640 1 -- 172.21.15.105:0/4224112853 wait complete. 2024-03-08T08:28:21.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.409+0000 7f62de7ac640 1 Processor -- start 2024-03-08T08:28:21.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62de7ac640 1 -- start start 2024-03-08T08:28:21.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62de7ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d814ce70 0x7f62d8152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62de7ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62d8152dc0 0x7f62d8153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62de7ac640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d806bd90 0x7f62d806c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62de7ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80d0310 con 0x7f62d814ce70 2024-03-08T08:28:21.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62de7ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80d0190 con 0x7f62d8152dc0 2024-03-08T08:28:21.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62de7ac640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80d0490 con 0x7f62d806bd90 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d814ce70 0x7f62d8152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d814ce70 0x7f62d8152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42536/0 (socket says 172.21.15.105:42536) 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.410+0000 7f62d7fff640 1 -- 172.21.15.105:0/4177212606 learned_addr learned my addr 172.21.15.105:0/4177212606 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d7fff640 1 -- 172.21.15.105:0/4177212606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d806bd90 msgr2=0x7f62d806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d7fff640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d806bd90 0x7f62d806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d7fff640 1 -- 172.21.15.105:0/4177212606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62d8152dc0 msgr2=0x7f62d8153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d7fff640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62d8152dc0 0x7f62d8153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d7fff640 1 -- 172.21.15.105:0/4177212606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80c5d20 con 0x7f62d814ce70 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d7fff640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d814ce70 0x7f62d8152880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007320 tx=0x7f62c80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d4ff9640 1 -- 172.21.15.105:0/4177212606 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c801a070 con 0x7f62d814ce70 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d4ff9640 1 -- 172.21.15.105:0/4177212606 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c800f9a0 con 0x7f62d814ce70 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62d4ff9640 1 -- 172.21.15.105:0/4177212606 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800fd70 con 0x7f62d814ce70 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d80c6000 con 0x7f62d814ce70 2024-03-08T08:28:21.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.411+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d80c65c0 con 0x7f62d814ce70 2024-03-08T08:28:21.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.412+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8075120 con 0x7f62d814ce70 2024-03-08T08:28:21.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.412+0000 7f62d4ff9640 1 -- 172.21.15.105:0/4177212606 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62c801d070 con 0x7f62d814ce70 2024-03-08T08:28:21.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.413+0000 7f62d4ff9640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f62c406b4d0 0x7f62c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.413+0000 7f62d4ff9640 1 -- 172.21.15.105:0/4177212606 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f62c8089c20 con 0x7f62d814ce70 2024-03-08T08:28:21.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.413+0000 7f62d77fe640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f62c406b4d0 0x7f62c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.413+0000 7f62d77fe640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f62c406b4d0 0x7f62c406d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f62d80cf410 tx=0x7f62cc009540 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.417+0000 7f62d4ff9640 1 -- 172.21.15.105:0/4177212606 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62c8058a20 con 0x7f62d814ce70 2024-03-08T08:28:21.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.625+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f62d80b31b0 con 0x7f62d814ce70 2024-03-08T08:28:21.626 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-03-08T08:28:21.625+0000 7f935598c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-08T08:28:21.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.626+0000 7f62d4ff9640 1 -- 172.21.15.105:0/4177212606 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f62c8058320 con 0x7f62d814ce70 2024-03-08T08:28:21.628 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-08T08:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f62c406b4d0 msgr2=0x7f62c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f62c406b4d0 0x7f62c406d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f62d80cf410 tx=0x7f62cc009540 comp rx=0 tx=0).stop 2024-03-08T08:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d814ce70 msgr2=0x7f62d8152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d814ce70 0x7f62d8152880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007320 tx=0x7f62c80077e0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 shutdown_connections 2024-03-08T08:28:21.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f62c406b4d0 0x7f62c406d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62d806bd90 0x7f62d806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62d8152dc0 0x7f62d8153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 --2- 172.21.15.105:0/4177212606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f62d814ce70 0x7f62d8152880 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 >> 172.21.15.105:0/4177212606 conn(0x7f62d80b36a0 msgr2=0x7f62d8154f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 shutdown_connections 2024-03-08T08:28:21.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.629+0000 7f62de7ac640 1 -- 172.21.15.105:0/4177212606 wait complete. 2024-03-08T08:28:21.635 INFO:tasks.ceph.mds.d.smithi144.stderr: -6868> 2024-03-08T08:27:43.633+0000 7fb72a52f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T08:28:21.643 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-08T08:28:21.668 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-03-08T08:28:21.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.744+0000 7f07151f4640 1 Processor -- start 2024-03-08T08:28:21.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.744+0000 7f07151f4640 1 -- start start 2024-03-08T08:28:21.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.744+0000 7f07151f4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07101469e0 0x7f071006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.744+0000 7f07151f4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07100adcd0 con 0x7f071006ac80 2024-03-08T08:28:21.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.744+0000 7f07151f4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071006deb0 con 0x7f07100ad870 2024-03-08T08:28:21.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.744+0000 7f07151f4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071006e190 con 0x7f07101469e0 2024-03-08T08:28:21.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.745+0000 7f070f59e640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f071006ac80 0x7f07100b4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42466/0 (socket says 172.21.15.105:42466) 2024-03-08T08:28:21.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.745+0000 7f070f59e640 1 -- 172.21.15.105:0/477285296 learned_addr learned my addr 172.21.15.105:0/477285296 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:21.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.745+0000 7f070ed9d640 1 --2- 172.21.15.105:0/477285296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07101469e0 0x7f071006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.745+0000 7f070dd9b640 1 -- 172.21.15.105:0/477285296 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766043122 0 0) 0x7f07100adcd0 con 0x7f071006ac80 2024-03-08T08:28:21.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.745+0000 7f070dd9b640 1 -- 172.21.15.105:0/477285296 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06f8003520 con 0x7f071006ac80 2024-03-08T08:28:21.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.745+0000 7f070ed9d640 1 -- 172.21.15.105:0/477285296 >> v1:172.21.15.144:6789/0 conn(0x7f07100ad870 legacy=0x7f071006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.745+0000 7f070ed9d640 1 -- 172.21.15.105:0/477285296 >> v1:172.21.15.105:6789/0 conn(0x7f071006ac80 legacy=0x7f07100b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.745+0000 7f070ed9d640 1 -- 172.21.15.105:0/477285296 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100b42f0 con 0x7f07101469e0 2024-03-08T08:28:21.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f070ed9d640 1 --2- 172.21.15.105:0/477285296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07101469e0 0x7f071006fd60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07000097d0 tx=0x7f070002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1b41819e66e26c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f070dd9b640 1 -- 172.21.15.105:0/477285296 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700035020 con 0x7f07101469e0 2024-03-08T08:28:21.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f070dd9b640 1 -- 172.21.15.105:0/477285296 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700031800 con 0x7f07101469e0 2024-03-08T08:28:21.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f070dd9b640 1 -- 172.21.15.105:0/477285296 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700031bb0 con 0x7f07101469e0 2024-03-08T08:28:21.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f07151f4640 1 -- 172.21.15.105:0/477285296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07101469e0 msgr2=0x7f071006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f07151f4640 1 --2- 172.21.15.105:0/477285296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07101469e0 0x7f071006fd60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07000097d0 tx=0x7f070002b8a0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f07151f4640 1 -- 172.21.15.105:0/477285296 shutdown_connections 2024-03-08T08:28:21.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f07151f4640 1 --2- 172.21.15.105:0/477285296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07101469e0 0x7f071006fd60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f07151f4640 1 -- 172.21.15.105:0/477285296 >> 172.21.15.105:0/477285296 conn(0x7f07100b2670 msgr2=0x7f07100b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:21.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f07151f4640 1 -- 172.21.15.105:0/477285296 shutdown_connections 2024-03-08T08:28:21.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.746+0000 7f07151f4640 1 -- 172.21.15.105:0/477285296 wait complete. 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.748+0000 7f07151f4640 1 Processor -- start 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.748+0000 7f07151f4640 1 -- start start 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f07151f4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f071006ac80 0x7f07100c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f07151f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07100ad870 0x7f0710132c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f07151f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07101469e0 0x7f0710133150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f07151f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0710146dc0 con 0x7f07100ad870 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f07151f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071006e470 con 0x7f07101469e0 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f07151f4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071006b110 con 0x7f071006ac80 2024-03-08T08:28:21.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07100ad870 0x7f0710132c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07100ad870 0x7f0710132c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42550/0 (socket says 172.21.15.105:42550) 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 -- 172.21.15.105:0/707380842 learned_addr learned my addr 172.21.15.105:0/707380842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070f59e640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07101469e0 0x7f0710133150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070ed9d640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f071006ac80 0x7f07100c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 -- 172.21.15.105:0/707380842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f071006ac80 msgr2=0x7f07100c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f071006ac80 0x7f07100c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 -- 172.21.15.105:0/707380842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07101469e0 msgr2=0x7f0710133150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07101469e0 0x7f0710133150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 -- 172.21.15.105:0/707380842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0710133880 con 0x7f07100ad870 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f070e59c640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07100ad870 0x7f0710132c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f070400c400 tx=0x7f070400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f06f77fe640 1 -- 172.21.15.105:0/707380842 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07040052d0 con 0x7f07100ad870 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f06f77fe640 1 -- 172.21.15.105:0/707380842 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0704005490 con 0x7f07100ad870 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07100d0130 con 0x7f07100ad870 2024-03-08T08:28:21.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.749+0000 7f06f77fe640 1 -- 172.21.15.105:0/707380842 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704007260 con 0x7f07100ad870 2024-03-08T08:28:21.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.750+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0710134100 con 0x7f07100ad870 2024-03-08T08:28:21.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.751+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0710146f40 con 0x7f07100ad870 2024-03-08T08:28:21.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.751+0000 7f06f77fe640 1 -- 172.21.15.105:0/707380842 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0704007400 con 0x7f07100ad870 2024-03-08T08:28:21.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.751+0000 7f06f77fe640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:21.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.751+0000 7f06f77fe640 1 -- 172.21.15.105:0/707380842 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f0704089730 con 0x7f07100ad870 2024-03-08T08:28:21.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.751+0000 7f070ed9d640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:21.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.752+0000 7f070ed9d640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f06fc06b4d0 0x7f06fc06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0700036000 tx=0x7f0700008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:21.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.755+0000 7f06f77fe640 1 -- 172.21.15.105:0/707380842 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0710146f40 con 0x7f07100ad870 2024-03-08T08:28:21.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:21.968+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0710134300 con 0x7f07100ad870 2024-03-08T08:28:22.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.628+0000 7f06f77fe640 1 -- 172.21.15.105:0/707380842 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0710134300 con 0x7f07100ad870 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f06fc06b4d0 msgr2=0x7f06fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f06fc06b4d0 0x7f06fc06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0700036000 tx=0x7f0700008030 comp rx=0 tx=0).stop 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07100ad870 msgr2=0x7f0710132c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07100ad870 0x7f0710132c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f070400c400 tx=0x7f070400c8c0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 shutdown_connections 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07101469e0 0x7f0710133150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07100ad870 0x7f0710132c10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 --2- 172.21.15.105:0/707380842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f071006ac80 0x7f07100c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.631+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 >> 172.21.15.105:0/707380842 conn(0x7f07100b2670 msgr2=0x7f071006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.632+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 shutdown_connections 2024-03-08T08:28:22.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.632+0000 7f07151f4640 1 -- 172.21.15.105:0/707380842 wait complete. 2024-03-08T08:28:22.646 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-08T08:28:22.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.753+0000 7fc67e683640 1 Processor -- start 2024-03-08T08:28:22.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.753+0000 7fc67e683640 1 -- start start 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.754+0000 7fc67e683640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc6780af160 0x7fc6780af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.754+0000 7fc67e683640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780afa80 0x7fc6780b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.754+0000 7fc67e683640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6780b0fc0 con 0x7fc6780b58f0 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.754+0000 7fc67e683640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6780b60e0 con 0x7fc6780af160 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.754+0000 7fc67e683640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780b1260 con 0x7fc6780afa80 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc6777fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780afa80 0x7fc6780b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc677fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc6780af160 0x7fc6780af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc6777fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780afa80 0x7fc6780b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:41502/0 (socket says 172.21.15.105:41502) 2024-03-08T08:28:22.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc6777fe640 1 -- 172.21.15.105:0/173459981 learned_addr learned my addr 172.21.15.105:0/173459981 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc67cc20640 1 --1- 172.21.15.105:0/173459981 >> v1:172.21.15.144:6789/0 conn(0x7fc6780b58f0 0x7fc6780b1d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:35708/0 (socket says 172.21.15.105:35708) 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc6777fe640 1 -- 172.21.15.105:0/173459981 >> v2:172.21.15.144:3300/0 conn(0x7fc6780af160 msgr2=0x7fc6780af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc6777fe640 1 --2- 172.21.15.105:0/173459981 >> v2:172.21.15.144:3300/0 conn(0x7fc6780af160 0x7fc6780af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc6777fe640 1 -- 172.21.15.105:0/173459981 >> v1:172.21.15.144:6789/0 conn(0x7fc6780b58f0 legacy=0x7fc6780b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc6777fe640 1 -- 172.21.15.105:0/173459981 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780b1540 con 0x7fc6780afa80 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.755+0000 7fc677fff640 1 --2- 172.21.15.105:0/173459981 >> v2:172.21.15.144:3300/0 conn(0x7fc6780af160 0x7fc6780af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc6777fe640 1 --2- 172.21.15.105:0/173459981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780afa80 0x7fc6780b53b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009870 tx=0x7fc66c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b95b5551511892a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc676ffd640 1 -- 172.21.15.105:0/173459981 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c035020 con 0x7fc6780afa80 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc676ffd640 1 -- 172.21.15.105:0/173459981 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c031640 con 0x7fc6780afa80 2024-03-08T08:28:22.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc676ffd640 1 -- 172.21.15.105:0/173459981 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c02d040 con 0x7fc6780afa80 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc67e683640 1 -- 172.21.15.105:0/173459981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780afa80 msgr2=0x7fc6780b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc67e683640 1 --2- 172.21.15.105:0/173459981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780afa80 0x7fc6780b53b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009870 tx=0x7fc66c02b8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc67e683640 1 -- 172.21.15.105:0/173459981 shutdown_connections 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc67e683640 1 --2- 172.21.15.105:0/173459981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780afa80 0x7fc6780b53b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc67e683640 1 --2- 172.21.15.105:0/173459981 >> v2:172.21.15.144:3300/0 conn(0x7fc6780af160 0x7fc6780af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc67e683640 1 -- 172.21.15.105:0/173459981 >> 172.21.15.105:0/173459981 conn(0x7fc67809aa00 msgr2=0x7fc67809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc67e683640 1 -- 172.21.15.105:0/173459981 shutdown_connections 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.756+0000 7fc67e683640 1 -- 172.21.15.105:0/173459981 wait complete. 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.758+0000 7fc67e683640 1 Processor -- start 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.758+0000 7fc67e683640 1 -- start start 2024-03-08T08:28:22.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.758+0000 7fc67e683640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780af160 0x7fc67809f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.758+0000 7fc67e683640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6780afa80 0x7fc6781f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.758+0000 7fc67e683640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6780b58f0 0x7fc678151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.758+0000 7fc67e683640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780b5d60 con 0x7fc6780afa80 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.758+0000 7fc67e683640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780afee0 con 0x7fc6780b58f0 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.758+0000 7fc67e683640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780b2450 con 0x7fc6780af160 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6780afa80 0x7fc6781f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6780afa80 0x7fc6781f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42556/0 (socket says 172.21.15.105:42556) 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 -- 172.21.15.105:0/2046473061 learned_addr learned my addr 172.21.15.105:0/2046473061 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc677fff640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780af160 0x7fc67809f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc67cc20640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6780b58f0 0x7fc678151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 -- 172.21.15.105:0/2046473061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780af160 msgr2=0x7fc67809f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:22.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780af160 0x7fc67809f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 -- 172.21.15.105:0/2046473061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6780b58f0 msgr2=0x7fc678151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6780b58f0 0x7fc678151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 -- 172.21.15.105:0/2046473061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678153b40 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc6777fe640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6780afa80 0x7fc6781f2820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc66c02b930 tx=0x7fc66c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc674ff9640 1 -- 172.21.15.105:0/2046473061 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c040070 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc674ff9640 1 -- 172.21.15.105:0/2046473061 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c03e070 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc674ff9640 1 -- 172.21.15.105:0/2046473061 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c035040 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67809eb60 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.759+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6781f2d60 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.760+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc678075200 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.761+0000 7fc674ff9640 1 -- 172.21.15.105:0/2046473061 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc66c038020 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.761+0000 7fc674ff9640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.761+0000 7fc674ff9640 1 -- 172.21.15.105:0/2046473061 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fc66c0abc60 con 0x7fc6780afa80 2024-03-08T08:28:22.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.761+0000 7fc677fff640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:22.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.761+0000 7fc677fff640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc66406b4d0 0x7fc66406d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc668009a30 tx=0x7fc668008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:22.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.765+0000 7fc674ff9640 1 -- 172.21.15.105:0/2046473061 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc66c07aaa0 con 0x7fc6780afa80 2024-03-08T08:28:22.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:22.936+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc67809e930 con 0x7fc6780afa80 2024-03-08T08:28:23.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.827+0000 7fc674ff9640 1 -- 172.21.15.105:0/2046473061 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc66c043e40 con 0x7fc6780afa80 2024-03-08T08:28:23.829 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-03-08T08:28:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc66406b4d0 msgr2=0x7fc66406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc66406b4d0 0x7fc66406d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc668009a30 tx=0x7fc668008040 comp rx=0 tx=0).stop 2024-03-08T08:28:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6780afa80 msgr2=0x7fc6781f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6780afa80 0x7fc6781f2820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc66c02b930 tx=0x7fc66c0046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 shutdown_connections 2024-03-08T08:28:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc66406b4d0 0x7fc66406d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6780b58f0 0x7fc678151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6780afa80 0x7fc6781f2820 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 --2- 172.21.15.105:0/2046473061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6780af160 0x7fc67809f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 >> 172.21.15.105:0/2046473061 conn(0x7fc67809aa00 msgr2=0x7fc678152660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 shutdown_connections 2024-03-08T08:28:23.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.831+0000 7fc67e683640 1 -- 172.21.15.105:0/2046473061 wait complete. 2024-03-08T08:28:23.846 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-08T08:28:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.954+0000 7f115cc10640 1 Processor -- start 2024-03-08T08:28:23.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.954+0000 7f115cc10640 1 -- start start 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115cc10640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f11580b5340 0x7f11580b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115cc10640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580b17f0 0x7f11580b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115cc10640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11580b60e0 con 0x7f115806c780 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115cc10640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11580b63c0 con 0x7f11580b5340 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115cc10640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b66a0 con 0x7f11580b17f0 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f115806c780 0x7f115806cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39118/0 (socket says 172.21.15.105:39118) 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115659c640 1 -- 172.21.15.105:0/118215886 learned_addr learned my addr 172.21.15.105:0/118215886 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f1155d9b640 1 --2- 172.21.15.105:0/118215886 >> v2:172.21.15.144:3300/0 conn(0x7f11580b5340 0x7f11580b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303860147 0 0) 0x7f11580b60e0 con 0x7f115806c780 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f1156d9d640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580b17f0 0x7f11580b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.955+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1140003520 con 0x7f115806c780 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3664725250 0 0) 0x7f1140003520 con 0x7f115806c780 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580b17f0 msgr2=0x7f11580b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580b17f0 0x7f11580b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 >> v2:172.21.15.144:3300/0 conn(0x7f11580b5340 msgr2=0x7f11580b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 --2- 172.21.15.105:0/118215886 >> v2:172.21.15.144:3300/0 conn(0x7f11580b5340 0x7f11580b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580b6980 con 0x7f115806c780 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f11480019f0 con 0x7f115806c780 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 >> v1:172.21.15.105:6789/0 conn(0x7f115806c780 legacy=0x7f115806cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1140003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580b17f0 0x7f1140004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1140004d40 0x7f11400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b63c0 con 0x7f11400039b0 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b66a0 con 0x7f1140004d40 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114000b9c0 con 0x7f11580b17f0 2024-03-08T08:28:23.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115659c640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1140003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f1156d9d640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1140004d40 0x7f11400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f1155d9b640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580b17f0 0x7f1140004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115659c640 1 -- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580b17f0 msgr2=0x7f1140004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115659c640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580b17f0 0x7f1140004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115659c640 1 -- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1140004d40 msgr2=0x7f11400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115659c640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1140004d40 0x7f11400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f115659c640 1 -- 172.21.15.105:0/118215886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140003520 con 0x7f11400039b0 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.956+0000 7f1156d9d640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1140004d40 0x7f11400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f115659c640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1140003dd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f114800a890 tx=0x7f1140005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d06c0cdd8aa15c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114800a4e0 con 0x7f11400039b0 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148034650 con 0x7f11400039b0 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f115559a640 1 -- 172.21.15.105:0/118215886 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148034a40 con 0x7f11400039b0 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f115cc10640 1 -- 172.21.15.105:0/118215886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 msgr2=0x7f1140003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f115cc10640 1 --2- 172.21.15.105:0/118215886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1140003dd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f114800a890 tx=0x7f1140005830 comp rx=0 tx=0).stop 2024-03-08T08:28:23.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f1156d9d640 1 -- 172.21.15.105:0/118215886 reap_dead start 2024-03-08T08:28:23.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f115cc10640 1 -- 172.21.15.105:0/118215886 shutdown_connections 2024-03-08T08:28:23.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.957+0000 7f115cc10640 1 -- 172.21.15.105:0/118215886 >> 172.21.15.105:0/118215886 conn(0x7f115809aa00 msgr2=0x7f115809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:23.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.958+0000 7f115cc10640 1 -- 172.21.15.105:0/118215886 shutdown_connections 2024-03-08T08:28:23.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.958+0000 7f115cc10640 1 -- 172.21.15.105:0/118215886 wait complete. 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.959+0000 7f115cc10640 1 Processor -- start 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.959+0000 7f115cc10640 1 -- start start 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115cc10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1158155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115cc10640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580cad20 0x7f11580afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115cc10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11580b0040 0x7f11580cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115cc10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b5ce0 con 0x7f11400039b0 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115cc10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115815e240 con 0x7f11580b0040 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115cc10640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b5e60 con 0x7f11580cad20 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1158155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1158155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34738/0 (socket says 172.21.15.105:34738) 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 -- 172.21.15.105:0/1828742772 learned_addr learned my addr 172.21.15.105:0/1828742772 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f1156d9d640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11580b0040 0x7f11580cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 -- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580cad20 msgr2=0x7f11580afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f1155d9b640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580cad20 0x7f11580afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580cad20 0x7f11580afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 -- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11580b0040 msgr2=0x7f11580cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11580b0040 0x7f11580cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 -- 172.21.15.105:0/1828742772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115815a640 con 0x7f11400039b0 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f1156d9d640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11580b0040 0x7f11580cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f1155d9b640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580cad20 0x7f11580afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115659c640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1158155750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1148000f80 tx=0x7f1148001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f1146ffd640 1 -- 172.21.15.105:0/1828742772 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148039040 con 0x7f11400039b0 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f1146ffd640 1 -- 172.21.15.105:0/1828742772 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148004b60 con 0x7f11400039b0 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115815a8e0 con 0x7f11400039b0 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f1146ffd640 1 -- 172.21.15.105:0/1828742772 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148034800 con 0x7f11400039b0 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.960+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115815aea0 con 0x7f11400039b0 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.961+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1158075200 con 0x7f11400039b0 2024-03-08T08:28:23.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.962+0000 7f1146ffd640 1 -- 172.21.15.105:0/1828742772 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11480349a0 con 0x7f11400039b0 2024-03-08T08:28:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.962+0000 7f1146ffd640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f113806b4d0 0x7f113806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.962+0000 7f1146ffd640 1 -- 172.21.15.105:0/1828742772 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f11480ac5f0 con 0x7f11400039b0 2024-03-08T08:28:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.962+0000 7f1155d9b640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f113806b4d0 0x7f113806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.963+0000 7f1155d9b640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f113806b4d0 0x7f113806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f115815db80 tx=0x7f114c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:23.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:23.967+0000 7f1146ffd640 1 -- 172.21.15.105:0/1828742772 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f114807b630 con 0x7f11400039b0 2024-03-08T08:28:24.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.136+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f115815ece0 con 0x7f11400039b0 2024-03-08T08:28:24.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.828+0000 7f1146ffd640 1 -- 172.21.15.105:0/1828742772 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f114807af30 con 0x7f11400039b0 2024-03-08T08:28:24.830 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-03-08T08:28:24.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f113806b4d0 msgr2=0x7f113806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:24.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f113806b4d0 0x7f113806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f115815db80 tx=0x7f114c008040 comp rx=0 tx=0).stop 2024-03-08T08:28:24.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 msgr2=0x7f1158155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:24.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1158155750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1148000f80 tx=0x7f1148001360 comp rx=0 tx=0).stop 2024-03-08T08:28:24.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 shutdown_connections 2024-03-08T08:28:24.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f113806b4d0 0x7f113806d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:24.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11580b0040 0x7f11580cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:24.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11580cad20 0x7f11580afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:24.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 --2- 172.21.15.105:0/1828742772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11400039b0 0x7f1158155750 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:24.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 >> 172.21.15.105:0/1828742772 conn(0x7f115809aa00 msgr2=0x7f115806d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:24.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 shutdown_connections 2024-03-08T08:28:24.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:24.831+0000 7f115cc10640 1 -- 172.21.15.105:0/1828742772 wait complete. 2024-03-08T08:28:24.848 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-08T08:28:24.848 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-08T08:28:24.849 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-08T08:28:24.849 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:28:24.849 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-08T08:28:24.849 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-08T08:28:24.849 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-08T08:28:24.849 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:28:24.850 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-08T08:28:24.850 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-08T08:28:24.850 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-08T08:28:24.850 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:28:24.850 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-08T08:28:24.850 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-08T08:28:24.851 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-08T08:28:24.851 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:28:24.858 INFO:tasks.ceph.mds.a:Stopped 2024-03-08T08:28:24.858 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-08T08:28:24.859 INFO:tasks.ceph.mds.c:Stopped 2024-03-08T08:28:24.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-08T08:28:24.861 INFO:tasks.ceph.mds.b:Stopped 2024-03-08T08:28:24.861 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-08T08:28:24.862 INFO:tasks.ceph.mds.d:Stopped 2024-03-08T08:28:24.862 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-08T08:28:24.901 INFO:tasks.ceph.mds.a:Started 2024-03-08T08:28:24.901 DEBUG:teuthology.parallel:result is None 2024-03-08T08:28:24.902 INFO:tasks.ceph.mds.c:Started 2024-03-08T08:28:24.902 DEBUG:teuthology.parallel:result is None 2024-03-08T08:28:24.904 INFO:tasks.ceph.mds.b:Started 2024-03-08T08:28:24.904 DEBUG:teuthology.parallel:result is None 2024-03-08T08:28:24.905 INFO:tasks.ceph.mds.d:Started 2024-03-08T08:28:24.905 DEBUG:teuthology.parallel:result is None 2024-03-08T08:28:24.905 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-08T08:28:24.977 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-03-08T08:28:24.978 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-03-08T08:28:24.982 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-03-08T08:28:24.983 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-03-08T08:28:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.009+0000 7fae07e15640 1 Processor -- start 2024-03-08T08:28:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.009+0000 7fae07e15640 1 -- start start 2024-03-08T08:28:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.010+0000 7fae07e15640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae000b4e40 0x7fae000b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.010+0000 7fae07e15640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae000b5a60 0x7fae0015c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.010+0000 7fae07e15640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0015f7b0 con 0x7fae0015bea0 2024-03-08T08:28:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.010+0000 7fae07e15640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0015e710 con 0x7fae000b5a60 2024-03-08T08:28:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.010+0000 7fae07e15640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0015e9f0 con 0x7fae000b4e40 2024-03-08T08:28:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.010+0000 7fae063b2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae000b5a60 0x7fae0015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.010+0000 7fae063b2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae000b5a60 0x7fae0015c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34766/0 (socket says 172.21.15.105:34766) 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.010+0000 7fae063b2640 1 -- 172.21.15.105:0/2858223564 learned_addr learned my addr 172.21.15.105:0/2858223564 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.011+0000 7fae053b0640 1 --2- 172.21.15.105:0/2858223564 >> v2:172.21.15.144:3300/0 conn(0x7fae000b4e40 0x7fae000b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.011+0000 7fae063b2640 1 -- 172.21.15.105:0/2858223564 >> v2:172.21.15.144:3300/0 conn(0x7fae000b4e40 msgr2=0x7fae000b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.011+0000 7fae063b2640 1 --2- 172.21.15.105:0/2858223564 >> v2:172.21.15.144:3300/0 conn(0x7fae000b4e40 0x7fae000b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.011+0000 7fae063b2640 1 -- 172.21.15.105:0/2858223564 >> v1:172.21.15.144:6789/0 conn(0x7fae0015bea0 legacy=0x7fae0015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.011+0000 7fae063b2640 1 -- 172.21.15.105:0/2858223564 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0015ecd0 con 0x7fae000b5a60 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.011+0000 7fae063b2640 1 --2- 172.21.15.105:0/2858223564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae0015c280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fae0006bad0 tx=0x7fadf802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21589e05fc7b653d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae04baf640 1 -- 172.21.15.105:0/2858223564 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8034070 con 0x7fae000b5a60 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae04baf640 1 -- 172.21.15.105:0/2858223564 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf8030860 con 0x7fae000b5a60 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae04baf640 1 -- 172.21.15.105:0/2858223564 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8030c30 con 0x7fae000b5a60 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae07e15640 1 -- 172.21.15.105:0/2858223564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 msgr2=0x7fae0015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae07e15640 1 --2- 172.21.15.105:0/2858223564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae0015c280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fae0006bad0 tx=0x7fadf802b9a0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae07e15640 1 -- 172.21.15.105:0/2858223564 shutdown_connections 2024-03-08T08:28:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae07e15640 1 --2- 172.21.15.105:0/2858223564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae0015c280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae07e15640 1 --2- 172.21.15.105:0/2858223564 >> v2:172.21.15.144:3300/0 conn(0x7fae000b4e40 0x7fae000b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae07e15640 1 -- 172.21.15.105:0/2858223564 >> 172.21.15.105:0/2858223564 conn(0x7fae0009aa00 msgr2=0x7fae0009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae07e15640 1 -- 172.21.15.105:0/2858223564 shutdown_connections 2024-03-08T08:28:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.012+0000 7fae07e15640 1 -- 172.21.15.105:0/2858223564 wait complete. 2024-03-08T08:28:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.013+0000 7fae07e15640 1 Processor -- start 2024-03-08T08:28:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.013+0000 7fae07e15640 1 -- start start 2024-03-08T08:28:25.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae07e15640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae000b4e40 0x7fae001468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae07e15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae000cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae07e15640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae0015bea0 0x7fae000ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae07e15640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0009cb00 con 0x7fae000b5a60 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae07e15640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015f290 con 0x7fae0015bea0 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae07e15640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015c7c0 con 0x7fae000b4e40 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae053b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae000cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae053b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae000cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34768/0 (socket says 172.21.15.105:34768) 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae053b0640 1 -- 172.21.15.105:0/3538143256 learned_addr learned my addr 172.21.15.105:0/3538143256 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae05bb1640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae000b4e40 0x7fae001468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.014+0000 7fae063b2640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae0015bea0 0x7fae000ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fae053b0640 1 -- 172.21.15.105:0/3538143256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae000b4e40 msgr2=0x7fae001468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fae053b0640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae000b4e40 0x7fae001468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fae053b0640 1 -- 172.21.15.105:0/3538143256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae0015bea0 msgr2=0x7fae000ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fae053b0640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae0015bea0 0x7fae000ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fae053b0640 1 -- 172.21.15.105:0/3538143256 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000cfcc0 con 0x7fae000b5a60 2024-03-08T08:28:25.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fae053b0640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae000cdc10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fadf000d830 tx=0x7fadf000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fadf67fc640 1 -- 172.21.15.105:0/3538143256 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000b6c0 con 0x7fae000b5a60 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fadf67fc640 1 -- 172.21.15.105:0/3538143256 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0019070 con 0x7fae000b5a60 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae000cff60 con 0x7fae000b5a60 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fadf67fc640 1 -- 172.21.15.105:0/3538143256 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000ba80 con 0x7fae000b5a60 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.015+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae00148dd0 con 0x7fae000b5a60 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.016+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae00067cb0 con 0x7fae000b5a60 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.016+0000 7fadf67fc640 1 -- 172.21.15.105:0/3538143256 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadf001e460 con 0x7fae000b5a60 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.016+0000 7fadf67fc640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fade406b4d0 0x7fade406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.017+0000 7fae05bb1640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fade406b4d0 0x7fade406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.017+0000 7fadf67fc640 1 -- 172.21.15.105:0/3538143256 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fadf008a3a0 con 0x7fae000b5a60 2024-03-08T08:28:25.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.017+0000 7fae05bb1640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fade406b4d0 0x7fade406d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0012e0 tx=0x7fadfc00e620 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:25.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.021+0000 7fadf67fc640 1 -- 172.21.15.105:0/3538143256 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae00067cb0 con 0x7fae000b5a60 2024-03-08T08:28:25.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.203+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fae00075120 con 0x7fae000b5a60 2024-03-08T08:28:25.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.847+0000 7fadf67fc640 1 -- 172.21.15.105:0/3538143256 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7fadf0058f60 con 0x7fae000b5a60 2024-03-08T08:28:25.849 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.849+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fade406b4d0 msgr2=0x7fade406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.849+0000 7fae07e15640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fade406b4d0 0x7fade406d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0012e0 tx=0x7fadfc00e620 comp rx=0 tx=0).stop 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.849+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 msgr2=0x7fae000cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.849+0000 7fae07e15640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae000cdc10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fadf000d830 tx=0x7fadf000dcf0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.849+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 shutdown_connections 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.850+0000 7fae07e15640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fade406b4d0 0x7fade406d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.850+0000 7fae07e15640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae0015bea0 0x7fae000ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.850+0000 7fae07e15640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae000b5a60 0x7fae000cdc10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.850+0000 7fae07e15640 1 --2- 172.21.15.105:0/3538143256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae000b4e40 0x7fae001468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.850+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 >> 172.21.15.105:0/3538143256 conn(0x7fae0009aa00 msgr2=0x7fae0006fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:25.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.850+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 shutdown_connections 2024-03-08T08:28:25.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.850+0000 7fae07e15640 1 -- 172.21.15.105:0/3538143256 wait complete. 2024-03-08T08:28:25.864 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-08T08:28:25.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.965+0000 7fd632ba0640 1 Processor -- start 2024-03-08T08:28:25.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.965+0000 7fd632ba0640 1 -- start start 2024-03-08T08:28:25.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd632ba0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 0x7fd62c0d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd632ba0640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd62c0b16a0 0x7fd62c0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd632ba0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62c0d29d0 con 0x7fd62c0b2af0 2024-03-08T08:28:25.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd632ba0640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c0b0690 con 0x7fd62c0b16a0 2024-03-08T08:28:25.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd632ba0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b2f20 con 0x7fd62c0d20b0 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd63139d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd62c0b2af0 0x7fd62c0cf180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39160/0 (socket says 172.21.15.105:39160) 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd63139d640 1 -- 172.21.15.105:0/790956630 learned_addr learned my addr 172.21.15.105:0/790956630 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd631b9e640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 0x7fd62c0d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd63239f640 1 --2- 172.21.15.105:0/790956630 >> v2:172.21.15.144:3300/0 conn(0x7fd62c0b16a0 0x7fd62c0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556500895 0 0) 0x7fd62c0d29d0 con 0x7fd62c0b2af0 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd618003520 con 0x7fd62c0b2af0 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3645633030 0 0) 0x7fd618003520 con 0x7fd62c0b2af0 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 msgr2=0x7fd62c0d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 0x7fd62c0d2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 >> v2:172.21.15.144:3300/0 conn(0x7fd62c0b16a0 msgr2=0x7fd62c0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 --2- 172.21.15.105:0/790956630 >> v2:172.21.15.144:3300/0 conn(0x7fd62c0b16a0 0x7fd62c0b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0cf890 con 0x7fd62c0b2af0 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fd61c001670 con 0x7fd62c0b2af0 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 >> v1:172.21.15.105:6789/0 conn(0x7fd62c0b2af0 legacy=0x7fd62c0cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618003a20 0x7fd618003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 0x7fd618004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd618004d40 0x7fd6180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b0690 con 0x7fd618004d40 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b2f20 con 0x7fd618003a20 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.966+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61800b9c0 con 0x7fd62c0d20b0 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd63239f640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd618004d40 0x7fd6180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd63139d640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 0x7fd618004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd631b9e640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618003a20 0x7fd618003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd63239f640 1 -- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 msgr2=0x7fd618004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd63239f640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 0x7fd618004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd63239f640 1 -- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618003a20 msgr2=0x7fd618003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd63239f640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618003a20 0x7fd618003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd63239f640 1 -- 172.21.15.105:0/790956630 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd618003520 con 0x7fd618004d40 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd63139d640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c0d20b0 0x7fd618004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.967+0000 7fd631b9e640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618003a20 0x7fd618003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd63239f640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd618004d40 0x7fd6180051c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd624009fc0 tx=0x7fd618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f099a0762ae8a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624039070 con 0x7fd618004d40 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62402cc60 con 0x7fd618004d40 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd630b9c640 1 -- 172.21.15.105:0/790956630 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624009c10 con 0x7fd618004d40 2024-03-08T08:28:25.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd632ba0640 1 -- 172.21.15.105:0/790956630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd618004d40 msgr2=0x7fd6180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd632ba0640 1 --2- 172.21.15.105:0/790956630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd618004d40 0x7fd6180051c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd624009fc0 tx=0x7fd618005830 comp rx=0 tx=0).stop 2024-03-08T08:28:25.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd63239f640 1 -- 172.21.15.105:0/790956630 reap_dead start 2024-03-08T08:28:25.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd632ba0640 1 -- 172.21.15.105:0/790956630 shutdown_connections 2024-03-08T08:28:25.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd632ba0640 1 -- 172.21.15.105:0/790956630 >> 172.21.15.105:0/790956630 conn(0x7fd62c09cda0 msgr2=0x7fd62c0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:25.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd632ba0640 1 -- 172.21.15.105:0/790956630 shutdown_connections 2024-03-08T08:28:25.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.968+0000 7fd632ba0640 1 -- 172.21.15.105:0/790956630 wait complete. 2024-03-08T08:28:25.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.969+0000 7fd632ba0640 1 Processor -- start 2024-03-08T08:28:25.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.969+0000 7fd632ba0640 1 -- start start 2024-03-08T08:28:25.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd632ba0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618004d40 0x7fd62c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd632ba0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c158b70 0x7fd62c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd632ba0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0b70f0 0x7fd62c0b7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd632ba0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b0ea0 con 0x7fd62c0b70f0 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd632ba0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b0bc0 con 0x7fd618004d40 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd632ba0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0b1180 con 0x7fd62c158b70 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63239f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0b70f0 0x7fd62c0b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63239f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0b70f0 0x7fd62c0b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34788/0 (socket says 172.21.15.105:34788) 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63239f640 1 -- 172.21.15.105:0/2022476787 learned_addr learned my addr 172.21.15.105:0/2022476787 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd631b9e640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618004d40 0x7fd62c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63139d640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c158b70 0x7fd62c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63239f640 1 -- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c158b70 msgr2=0x7fd62c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63239f640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c158b70 0x7fd62c0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63239f640 1 -- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618004d40 msgr2=0x7fd62c0b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63239f640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618004d40 0x7fd62c0b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd63239f640 1 -- 172.21.15.105:0/2022476787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0cc130 con 0x7fd62c0b70f0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.970+0000 7fd631b9e640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618004d40 0x7fd62c0b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.971+0000 7fd63239f640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0b70f0 0x7fd62c0b7570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd62400e230 tx=0x7fd624009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.971+0000 7fd6227fc640 1 -- 172.21.15.105:0/2022476787 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624039030 con 0x7fd62c0b70f0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.971+0000 7fd6227fc640 1 -- 172.21.15.105:0/2022476787 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62403a040 con 0x7fd62c0b70f0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.971+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c0b9100 con 0x7fd62c0b70f0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.971+0000 7fd6227fc640 1 -- 172.21.15.105:0/2022476787 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624003990 con 0x7fd62c0b70f0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.971+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c0b68f0 con 0x7fd62c0b70f0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.972+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c1614a0 con 0x7fd62c0b70f0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.972+0000 7fd6227fc640 1 -- 172.21.15.105:0/2022476787 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd624045020 con 0x7fd62c0b70f0 2024-03-08T08:28:25.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.973+0000 7fd6227fc640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd61006b4d0 0x7fd61006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:25.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.973+0000 7fd6227fc640 1 -- 172.21.15.105:0/2022476787 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd6240abce0 con 0x7fd62c0b70f0 2024-03-08T08:28:25.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.973+0000 7fd631b9e640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd61006b4d0 0x7fd61006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:25.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.973+0000 7fd631b9e640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd61006b4d0 0x7fd61006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd628009a50 tx=0x7fd628008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:25.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:25.977+0000 7fd6227fc640 1 -- 172.21.15.105:0/2022476787 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd62407af40 con 0x7fd62c0b70f0 2024-03-08T08:28:26.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.214+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd62c0d1e90 con 0x7fd62c0b70f0 2024-03-08T08:28:26.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.214+0000 7fd6227fc640 1 -- 172.21.15.105:0/2022476787 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd62407a840 con 0x7fd62c0b70f0 2024-03-08T08:28:26.216 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+plo/lhChAAG3AxM32YObwo6H9eS31x1g==", 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-08T08:28:26.217 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+pl1r+gFhAA7lND37bRmxyruezGya8zZg==", 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-08T08:28:26.218 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+pla7qwERAAeVrSqF6zglXuWKRqC1e9hg==", 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.219 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+pls20EHhAAmSsOB414olq6Vz1VcC/uvQ==", 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDFy+pl8FscBBAA8TFf1gnifLU4Bdu/8Zp2AQ==", 2024-03-08T08:28:26.220 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDIy+plYj2nAhAAA+swRGEJkSYG/A2Bjeyztg==", 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.221 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDLy+plLNvNABAA+ErHHuno91i45lPDcQVlbA==", 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.222 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDNy+plG8YpOxAAolVB6FJlT8X/cuKRa2E9+g==", 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDUy+plKvXrBxAAWA+vmWj1l68pyQ81i6Gxhg==", 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.223 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDXy+pl+R1NBhAARFNkrHGsiUCbQyobuuoWTg==", 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.224 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDay+plTd5DBBAAMgKGvfqMAw8OyFpZcB5ryg==", 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDdy+plum0eAhAAV2lbgyOIrv8Z6m3tdxNdSA==", 2024-03-08T08:28:26.225 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+pl4CRKIhAAtr/P93sD/1absmF733EcnQ==", 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-03-08T08:28:26.226 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+plwux9KxAArAvxh6JfHfMYd8kHKMOfXw==", 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.227 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+pl2bATKBAA1L5Hmjhd3LmVbLiOXqyCcg==", 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2024-03-08T08:28:26.228 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+pl/WFHMRAA5pEX7jDEu6LBea+B7ULUiQ==", 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDAy+plqSjPKRAAVnWX9aNr8Uf+VOPpVqgAKw==", 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-03-08T08:28:26.229 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mds", 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDiy+plnEqgNhAAZJodFhobf11urEQFYcIwag==", 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mds" 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.230 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mgr", 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDiy+plR1KgNhAAoBnHt+SDHUJOVF9+NfCPPw==", 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-osd", 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDiy+pl9ligNhAAXMXUIP8PZQ1S2QGF0wbIbw==", 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-osd" 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd", 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDiy+plrl+gNhAAHslMHEPui/+PjNNrmBi64Q==", 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDiy+pljWagNhAA9Wmr+8mV33+kDiPG5LS20g==", 2024-03-08T08:28:26.232 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rgw", 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDiy+plkG2gNhAA66NY1bWhmUSeY91agWlBgQ==", 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2024-03-08T08:28:26.233 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+plV4P+BhAAs7ZptbBbuQM8B7SnOAGOnQ==", 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDBy+pleaZMAxAAaBcd2a/W/2YV1SCHxNNT4Q==", 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-03-08T08:28:26.234 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2024-03-08T08:28:26.235 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-03-08T08:28:26.235 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.235 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:26.235 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-08T08:28:26.235 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:28:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd61006b4d0 msgr2=0x7fd61006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd61006b4d0 0x7fd61006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd628009a50 tx=0x7fd628008040 comp rx=0 tx=0).stop 2024-03-08T08:28:26.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0b70f0 msgr2=0x7fd62c0b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0b70f0 0x7fd62c0b7570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd62400e230 tx=0x7fd624009750 comp rx=0 tx=0).stop 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 shutdown_connections 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd61006b4d0 0x7fd61006d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd62c0b70f0 0x7fd62c0b7570 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd62c158b70 0x7fd62c0cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 --2- 172.21.15.105:0/2022476787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd618004d40 0x7fd62c0b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 >> 172.21.15.105:0/2022476787 conn(0x7fd62c09cda0 msgr2=0x7fd62c06b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 shutdown_connections 2024-03-08T08:28:26.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.216+0000 7fd632ba0640 1 -- 172.21.15.105:0/2022476787 wait complete. 2024-03-08T08:28:26.238 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-08T08:28:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.337+0000 7fa1a1381640 1 Processor -- start 2024-03-08T08:28:26.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.337+0000 7fa1a1381640 1 -- start start 2024-03-08T08:28:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa1a1381640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa19c161850 0x7fa19c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa1a1381640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19c0d48a0 con 0x7fa19c161470 2024-03-08T08:28:26.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa1a1381640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19c0d4b80 con 0x7fa19c06bcb0 2024-03-08T08:28:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa1a1381640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa19c06c170 con 0x7fa19c161850 2024-03-08T08:28:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa19b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa19c161850 0x7fa19c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa19affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa19c161470 0x7fa19c06b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39172/0 (socket says 172.21.15.105:39172) 2024-03-08T08:28:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa19affd640 1 -- 172.21.15.105:0/3400213967 learned_addr learned my addr 172.21.15.105:0/3400213967 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa19b7fe640 1 -- 172.21.15.105:0/3400213967 >> v1:172.21.15.144:6789/0 conn(0x7fa19c06bcb0 legacy=0x7fa19c0b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa19b7fe640 1 -- 172.21.15.105:0/3400213967 >> v1:172.21.15.105:6789/0 conn(0x7fa19c161470 legacy=0x7fa19c06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa19b7fe640 1 -- 172.21.15.105:0/3400213967 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c06c450 con 0x7fa19c161850 2024-03-08T08:28:26.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.338+0000 7fa199ffb640 1 -- 172.21.15.105:0/3400213967 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495290076 0 0) 0x7fa19c0d48a0 con 0x7fa19c161470 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.339+0000 7fa19b7fe640 1 --2- 172.21.15.105:0/3400213967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c161850 0x7fa19c06c830 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa194009d90 tx=0x7fa19402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29619359649c121e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.339+0000 7fa199ffb640 1 -- 172.21.15.105:0/3400213967 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194039070 con 0x7fa19c161850 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.339+0000 7fa199ffb640 1 -- 172.21.15.105:0/3400213967 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19402cbd0 con 0x7fa19c161850 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.339+0000 7fa199ffb640 1 -- 172.21.15.105:0/3400213967 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194034440 con 0x7fa19c161850 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.340+0000 7fa1a1381640 1 -- 172.21.15.105:0/3400213967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c161850 msgr2=0x7fa19c06c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.340+0000 7fa1a1381640 1 --2- 172.21.15.105:0/3400213967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c161850 0x7fa19c06c830 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa194009d90 tx=0x7fa19402be20 comp rx=0 tx=0).stop 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.340+0000 7fa1a1381640 1 -- 172.21.15.105:0/3400213967 shutdown_connections 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.340+0000 7fa1a1381640 1 --2- 172.21.15.105:0/3400213967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c161850 0x7fa19c06c830 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.340+0000 7fa1a1381640 1 -- 172.21.15.105:0/3400213967 >> 172.21.15.105:0/3400213967 conn(0x7fa19c0d3ce0 msgr2=0x7fa19c0d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:26.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.340+0000 7fa1a1381640 1 -- 172.21.15.105:0/3400213967 shutdown_connections 2024-03-08T08:28:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.340+0000 7fa1a1381640 1 -- 172.21.15.105:0/3400213967 wait complete. 2024-03-08T08:28:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.341+0000 7fa1a1381640 1 Processor -- start 2024-03-08T08:28:26.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.341+0000 7fa1a1381640 1 -- start start 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1a1381640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c06bcb0 0x7fa19c14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1a1381640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa19c161470 0x7fa19c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1a1381640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa19c161850 0x7fa19c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1a1381640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c06d4f0 con 0x7fa19c06bcb0 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1a1381640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c06d370 con 0x7fa19c161470 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1a1381640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c06d670 con 0x7fa19c161850 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c06bcb0 0x7fa19c14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c06bcb0 0x7fa19c14f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34814/0 (socket says 172.21.15.105:34814) 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 -- 172.21.15.105:0/2098954654 learned_addr learned my addr 172.21.15.105:0/2098954654 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19a7fc640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa19c161470 0x7fa19c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19b7fe640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa19c161850 0x7fa19c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 -- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa19c161850 msgr2=0x7fa19c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa19c161850 0x7fa19c0cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 -- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa19c161470 msgr2=0x7fa19c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa19c161470 0x7fa19c0cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 -- 172.21.15.105:0/2098954654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c0b7760 con 0x7fa19c06bcb0 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19b7fe640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa19c161850 0x7fa19c0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa19affd640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c06bcb0 0x7fa19c14f000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00c4d0 tx=0x7fa18c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1837fe640 1 -- 172.21.15.105:0/2098954654 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c018070 con 0x7fa19c06bcb0 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1837fe640 1 -- 172.21.15.105:0/2098954654 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c007500 con 0x7fa19c06bcb0 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19c0b7a40 con 0x7fa19c06bcb0 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.342+0000 7fa1837fe640 1 -- 172.21.15.105:0/2098954654 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c004a20 con 0x7fa19c06bcb0 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.343+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19c0b8000 con 0x7fa19c06bcb0 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.344+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19c075120 con 0x7fa19c06bcb0 2024-03-08T08:28:26.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.344+0000 7fa1837fe640 1 -- 172.21.15.105:0/2098954654 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa18c01b070 con 0x7fa19c06bcb0 2024-03-08T08:28:26.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.344+0000 7fa1837fe640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa18806b500 0x7fa18806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.344+0000 7fa1837fe640 1 -- 172.21.15.105:0/2098954654 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa18c089190 con 0x7fa19c06bcb0 2024-03-08T08:28:26.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.344+0000 7fa19a7fc640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa18806b500 0x7fa18806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.345+0000 7fa19a7fc640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa18806b500 0x7fa18806d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0cf970 tx=0x7fa190009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:26.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.349+0000 7fa1837fe640 1 -- 172.21.15.105:0/2098954654 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa18c058360 con 0x7fa19c06bcb0 2024-03-08T08:28:26.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.583+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fa19c0d40c0 con 0x7fa19c06bcb0 2024-03-08T08:28:26.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.585+0000 7fa1837fe640 1 -- 172.21.15.105:0/2098954654 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa18c057c60 con 0x7fa19c06bcb0 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa18806b500 msgr2=0x7fa18806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa18806b500 0x7fa18806d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0cf970 tx=0x7fa190009040 comp rx=0 tx=0).stop 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c06bcb0 msgr2=0x7fa19c14f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c06bcb0 0x7fa19c14f000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00c4d0 tx=0x7fa18c00c990 comp rx=0 tx=0).stop 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 shutdown_connections 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa18806b500 0x7fa18806d9c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa19c161850 0x7fa19c0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa19c161470 0x7fa19c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 --2- 172.21.15.105:0/2098954654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa19c06bcb0 0x7fa19c14f000 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 >> 172.21.15.105:0/2098954654 conn(0x7fa19c0d3ce0 msgr2=0x7fa19c15def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:26.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 shutdown_connections 2024-03-08T08:28:26.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.588+0000 7fa1a1381640 1 -- 172.21.15.105:0/2098954654 wait complete. 2024-03-08T08:28:26.600 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-08T08:28:26.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.699+0000 7f555b2fb640 1 Processor -- start 2024-03-08T08:28:26.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.699+0000 7f555b2fb640 1 -- start start 2024-03-08T08:28:26.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f555b2fb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5554072440 0x7f5554071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f555b2fb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f55540b4d00 0x7f555415af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f555b2fb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555415b940 con 0x7f555415b4a0 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f555b2fb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555415bc20 con 0x7f5554072440 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f555b2fb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5554070840 con 0x7f55540b4d00 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f5559097640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5554072440 0x7f5554071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f5559097640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5554072440 0x7f5554071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34826/0 (socket says 172.21.15.105:34826) 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f5559097640 1 -- 172.21.15.105:0/16306515 learned_addr learned my addr 172.21.15.105:0/16306515 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f5558896640 1 --2- 172.21.15.105:0/16306515 >> v2:172.21.15.144:3300/0 conn(0x7f55540b4d00 0x7f555415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f5559097640 1 -- 172.21.15.105:0/16306515 >> v2:172.21.15.144:3300/0 conn(0x7f55540b4d00 msgr2=0x7f555415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f5559097640 1 --2- 172.21.15.105:0/16306515 >> v2:172.21.15.144:3300/0 conn(0x7f55540b4d00 0x7f555415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f5559097640 1 -- 172.21.15.105:0/16306515 >> v1:172.21.15.144:6789/0 conn(0x7f555415b4a0 legacy=0x7f555406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.700+0000 7f5559097640 1 -- 172.21.15.105:0/16306515 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554070b20 con 0x7f5554072440 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.701+0000 7f5559097640 1 --2- 172.21.15.105:0/16306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5554072440 0x7f5554071020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f55540cf2b0 tx=0x7f554c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653481ee1fac08da server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.701+0000 7f554bfff640 1 -- 172.21.15.105:0/16306515 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c02cb20 con 0x7f5554072440 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.701+0000 7f554bfff640 1 -- 172.21.15.105:0/16306515 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c02ccc0 con 0x7f5554072440 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.701+0000 7f554bfff640 1 -- 172.21.15.105:0/16306515 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c034610 con 0x7f5554072440 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.702+0000 7f555b2fb640 1 -- 172.21.15.105:0/16306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5554072440 msgr2=0x7f5554071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.702+0000 7f555b2fb640 1 --2- 172.21.15.105:0/16306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5554072440 0x7f5554071020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f55540cf2b0 tx=0x7f554c02bf80 comp rx=0 tx=0).stop 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.702+0000 7f555b2fb640 1 -- 172.21.15.105:0/16306515 shutdown_connections 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.702+0000 7f555b2fb640 1 --2- 172.21.15.105:0/16306515 >> v2:172.21.15.144:3300/0 conn(0x7f55540b4d00 0x7f555415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.702+0000 7f555b2fb640 1 --2- 172.21.15.105:0/16306515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5554072440 0x7f5554071020 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.702+0000 7f555b2fb640 1 -- 172.21.15.105:0/16306515 >> 172.21.15.105:0/16306515 conn(0x7f55540b1740 msgr2=0x7f55540b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:26.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.702+0000 7f555b2fb640 1 -- 172.21.15.105:0/16306515 shutdown_connections 2024-03-08T08:28:26.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.702+0000 7f555b2fb640 1 -- 172.21.15.105:0/16306515 wait complete. 2024-03-08T08:28:26.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.703+0000 7f555b2fb640 1 Processor -- start 2024-03-08T08:28:26.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.703+0000 7f555b2fb640 1 -- start start 2024-03-08T08:28:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f555b2fb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5554072440 0x7f5554154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f555b2fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55540b4d00 0x7f55541e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f555b2fb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f555415b4a0 0x7f555415a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f555b2fb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540729a0 con 0x7f55540b4d00 2024-03-08T08:28:26.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f555b2fb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5554072820 con 0x7f5554072440 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f555b2fb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555406e7b0 con 0x7f555415b4a0 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55540b4d00 0x7f55541e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55540b4d00 0x7f55541e51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34838/0 (socket says 172.21.15.105:34838) 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 -- 172.21.15.105:0/3039984009 learned_addr learned my addr 172.21.15.105:0/3039984009 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5559097640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5554072440 0x7f5554154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5559898640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f555415b4a0 0x7f555415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 -- 172.21.15.105:0/3039984009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f555415b4a0 msgr2=0x7f555415a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f555415b4a0 0x7f555415a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 -- 172.21.15.105:0/3039984009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5554072440 msgr2=0x7f5554154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5554072440 0x7f5554154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 -- 172.21.15.105:0/3039984009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555415acf0 con 0x7f55540b4d00 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5558896640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55540b4d00 0x7f55541e51d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f554400d7d0 tx=0x7f554400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5549ffb640 1 -- 172.21.15.105:0/3039984009 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400b6c0 con 0x7f55540b4d00 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5549ffb640 1 -- 172.21.15.105:0/3039984009 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544019070 con 0x7f55540b4d00 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55541e4410 con 0x7f55540b4d00 2024-03-08T08:28:26.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.704+0000 7f5549ffb640 1 -- 172.21.15.105:0/3039984009 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400bac0 con 0x7f55540b4d00 2024-03-08T08:28:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.705+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55541e49d0 con 0x7f55540b4d00 2024-03-08T08:28:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.706+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5554141920 con 0x7f55540b4d00 2024-03-08T08:28:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.706+0000 7f5549ffb640 1 -- 172.21.15.105:0/3039984009 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f554400bc60 con 0x7f55540b4d00 2024-03-08T08:28:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.706+0000 7f5549ffb640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f553806b4d0 0x7f553806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.706+0000 7f5549ffb640 1 -- 172.21.15.105:0/3039984009 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f554408a2d0 con 0x7f55540b4d00 2024-03-08T08:28:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.706+0000 7f5559097640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f553806b4d0 0x7f553806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:26.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.707+0000 7f5559097640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f553806b4d0 0x7f553806d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f554c035000 tx=0x7f554c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:26.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.711+0000 7f5549ffb640 1 -- 172.21.15.105:0/3039984009 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55440594a0 con 0x7f55540b4d00 2024-03-08T08:28:26.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.950+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f5554155350 con 0x7f55540b4d00 2024-03-08T08:28:26.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.952+0000 7f5549ffb640 1 -- 172.21.15.105:0/3039984009 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5544058da0 con 0x7f55540b4d00 2024-03-08T08:28:26.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f553806b4d0 msgr2=0x7f553806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f553806b4d0 0x7f553806d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f554c035000 tx=0x7f554c030000 comp rx=0 tx=0).stop 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55540b4d00 msgr2=0x7f55541e51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55540b4d00 0x7f55541e51d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f554400d7d0 tx=0x7f554400dc90 comp rx=0 tx=0).stop 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 shutdown_connections 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f553806b4d0 0x7f553806d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f555415b4a0 0x7f555415a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55540b4d00 0x7f55541e51d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 --2- 172.21.15.105:0/3039984009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5554072440 0x7f5554154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 >> 172.21.15.105:0/3039984009 conn(0x7f55540b1740 msgr2=0x7f5554150170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 shutdown_connections 2024-03-08T08:28:26.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:26.954+0000 7f555b2fb640 1 -- 172.21.15.105:0/3039984009 wait complete. 2024-03-08T08:28:26.967 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-08T08:28:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.066+0000 7f0852508640 1 Processor -- start 2024-03-08T08:28:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.066+0000 7f0852508640 1 -- start start 2024-03-08T08:28:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.066+0000 7f0852508640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f084c072440 0x7f084c071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.066+0000 7f0852508640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f084c0b4d00 0x7f084c15af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.066+0000 7f0852508640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f084c15b940 con 0x7f084c15b4a0 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.066+0000 7f0852508640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084c15bc20 con 0x7f084c0b4d00 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.066+0000 7f0852508640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084c070840 con 0x7f084c072440 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f084b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f084c0b4d00 0x7f084c15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f0850aa5640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f084c15b4a0 0x7f084c06e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39182/0 (socket says 172.21.15.105:39182) 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f0850aa5640 1 -- 172.21.15.105:0/2622873066 learned_addr learned my addr 172.21.15.105:0/2622873066 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f084bfff640 1 --2- 172.21.15.105:0/2622873066 >> v2:172.21.15.144:3300/0 conn(0x7f084c072440 0x7f084c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f084b7fe640 1 -- 172.21.15.105:0/2622873066 >> v2:172.21.15.144:3300/0 conn(0x7f084c072440 msgr2=0x7f084c071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f084b7fe640 1 --2- 172.21.15.105:0/2622873066 >> v2:172.21.15.144:3300/0 conn(0x7f084c072440 0x7f084c071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f084b7fe640 1 -- 172.21.15.105:0/2622873066 >> v1:172.21.15.105:6789/0 conn(0x7f084c15b4a0 legacy=0x7f084c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f084b7fe640 1 -- 172.21.15.105:0/2622873066 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c070b20 con 0x7f084c0b4d00 2024-03-08T08:28:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.067+0000 7f084affd640 1 -- 172.21.15.105:0/2622873066 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375990455 0 0) 0x7f084c15b940 con 0x7f084c15b4a0 2024-03-08T08:28:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f084b7fe640 1 --2- 172.21.15.105:0/2622873066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c0b4d00 0x7f084c15af60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f084c06cae0 tx=0x7f084002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220e11ce8b2e776c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f084affd640 1 -- 172.21.15.105:0/2622873066 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840035020 con 0x7f084c0b4d00 2024-03-08T08:28:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f084affd640 1 -- 172.21.15.105:0/2622873066 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0840031830 con 0x7f084c0b4d00 2024-03-08T08:28:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f084affd640 1 -- 172.21.15.105:0/2622873066 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840031c00 con 0x7f084c0b4d00 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f0852508640 1 -- 172.21.15.105:0/2622873066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c0b4d00 msgr2=0x7f084c15af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f0852508640 1 --2- 172.21.15.105:0/2622873066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c0b4d00 0x7f084c15af60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f084c06cae0 tx=0x7f084002b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f0852508640 1 -- 172.21.15.105:0/2622873066 shutdown_connections 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f0852508640 1 --2- 172.21.15.105:0/2622873066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c0b4d00 0x7f084c15af60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f0852508640 1 --2- 172.21.15.105:0/2622873066 >> v2:172.21.15.144:3300/0 conn(0x7f084c072440 0x7f084c071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f0852508640 1 -- 172.21.15.105:0/2622873066 >> 172.21.15.105:0/2622873066 conn(0x7f084c0b1740 msgr2=0x7f084c0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f0852508640 1 -- 172.21.15.105:0/2622873066 shutdown_connections 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.068+0000 7f0852508640 1 -- 172.21.15.105:0/2622873066 wait complete. 2024-03-08T08:28:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.069+0000 7f0852508640 1 Processor -- start 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f0852508640 1 -- start start 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f0852508640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c072440 0x7f084c1f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f0852508640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f084c0b4d00 0x7f084c1557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f0852508640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f084c15b4a0 0x7f084c14d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f0852508640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c0729a0 con 0x7f084c072440 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f0852508640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c072820 con 0x7f084c15b4a0 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f0852508640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084c06e7b0 con 0x7f084c0b4d00 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f084bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c072440 0x7f084c1f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f084bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c072440 0x7f084c1f0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34866/0 (socket says 172.21.15.105:34866) 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.070+0000 7f084bfff640 1 -- 172.21.15.105:0/1289336320 learned_addr learned my addr 172.21.15.105:0/1289336320 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f0850aa5640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f084c15b4a0 0x7f084c14d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f084b7fe640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f084c0b4d00 0x7f084c1557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f084bfff640 1 -- 172.21.15.105:0/1289336320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f084c0b4d00 msgr2=0x7f084c1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f084bfff640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f084c0b4d00 0x7f084c1557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f084bfff640 1 -- 172.21.15.105:0/1289336320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f084c15b4a0 msgr2=0x7f084c14d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f084bfff640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f084c15b4a0 0x7f084c14d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f084bfff640 1 -- 172.21.15.105:0/1289336320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c13beb0 con 0x7f084c072440 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f084bfff640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c072440 0x7f084c1f0e90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f083c00c7f0 tx=0x7f083c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f0848ff9640 1 -- 172.21.15.105:0/1289336320 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c015070 con 0x7f084c072440 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f0848ff9640 1 -- 172.21.15.105:0/1289336320 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c004240 con 0x7f084c072440 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f0848ff9640 1 -- 172.21.15.105:0/1289336320 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c007620 con 0x7f084c072440 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084c13c190 con 0x7f084c072440 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.071+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084c13c750 con 0x7f084c072440 2024-03-08T08:28:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.072+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084c075120 con 0x7f084c072440 2024-03-08T08:28:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.072+0000 7f0848ff9640 1 -- 172.21.15.105:0/1289336320 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f083c0079d0 con 0x7f084c072440 2024-03-08T08:28:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.073+0000 7f0848ff9640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f083806b4d0 0x7f083806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.073+0000 7f084b7fe640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f083806b4d0 0x7f083806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.073+0000 7f0848ff9640 1 -- 172.21.15.105:0/1289336320 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f083c08a2a0 con 0x7f084c072440 2024-03-08T08:28:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.073+0000 7f084b7fe640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f083806b4d0 0x7f083806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0840036000 tx=0x7f0840008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.077+0000 7f0848ff9640 1 -- 172.21.15.105:0/1289336320 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f083c059500 con 0x7f084c072440 2024-03-08T08:28:27.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.312+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f084c0b47d0 con 0x7f084c072440 2024-03-08T08:28:27.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.314+0000 7f0848ff9640 1 -- 172.21.15.105:0/1289336320 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f083c058e00 con 0x7f084c072440 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f083806b4d0 msgr2=0x7f083806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f083806b4d0 0x7f083806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0840036000 tx=0x7f0840008030 comp rx=0 tx=0).stop 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c072440 msgr2=0x7f084c1f0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c072440 0x7f084c1f0e90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f083c00c7f0 tx=0x7f083c00ccb0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 shutdown_connections 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f083806b4d0 0x7f083806d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f084c15b4a0 0x7f084c14d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f084c0b4d00 0x7f084c1557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 --2- 172.21.15.105:0/1289336320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f084c072440 0x7f084c1f0e90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 >> 172.21.15.105:0/1289336320 conn(0x7f084c0b1740 msgr2=0x7f084c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 shutdown_connections 2024-03-08T08:28:27.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.316+0000 7f0852508640 1 -- 172.21.15.105:0/1289336320 wait complete. 2024-03-08T08:28:27.328 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-08T08:28:27.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.427+0000 7f1ea56b5640 1 Processor -- start 2024-03-08T08:28:27.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.427+0000 7f1ea56b5640 1 -- start start 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1ea56b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea0161470 0x7f1ea006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1ea56b5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ea0161850 0x7f1ea00d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1ea56b5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea00d48a0 con 0x7f1ea006d370 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1ea56b5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea00d2ba0 con 0x7f1ea0161850 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1ea56b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00d4b80 con 0x7f1ea0161470 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9e7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ea0161850 0x7f1ea00d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9e7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1ea0161850 0x7f1ea00d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34880/0 (socket says 172.21.15.105:34880) 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9e7fc640 1 -- 172.21.15.105:0/1222779829 learned_addr learned my addr 172.21.15.105:0/1222779829 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9effd640 1 --2- 172.21.15.105:0/1222779829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea0161470 0x7f1ea006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9e7fc640 1 -- 172.21.15.105:0/1222779829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea0161470 msgr2=0x7f1ea006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9e7fc640 1 --2- 172.21.15.105:0/1222779829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea0161470 0x7f1ea006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9e7fc640 1 -- 172.21.15.105:0/1222779829 >> v1:172.21.15.144:6789/0 conn(0x7f1ea006d370 legacy=0x7f1ea006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:27.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9e7fc640 1 -- 172.21.15.105:0/1222779829 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea006c170 con 0x7f1ea0161850 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.428+0000 7f1e9effd640 1 --2- 172.21.15.105:0/1222779829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea0161470 0x7f1ea006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.429+0000 7f1e9e7fc640 1 --2- 172.21.15.105:0/1222779829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea00d2660 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009870 tx=0x7f1e9402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cc71881333d29d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.429+0000 7f1e9dffb640 1 -- 172.21.15.105:0/1222779829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e940317c0 con 0x7f1ea0161850 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.429+0000 7f1e9dffb640 1 -- 172.21.15.105:0/1222779829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e94037070 con 0x7f1ea0161850 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.429+0000 7f1e9dffb640 1 -- 172.21.15.105:0/1222779829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94031b80 con 0x7f1ea0161850 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.429+0000 7f1ea56b5640 1 -- 172.21.15.105:0/1222779829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 msgr2=0x7f1ea00d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.429+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/1222779829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea00d2660 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009870 tx=0x7f1e9402b860 comp rx=0 tx=0).stop 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.430+0000 7f1ea56b5640 1 -- 172.21.15.105:0/1222779829 shutdown_connections 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.430+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/1222779829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea00d2660 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.430+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/1222779829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea0161470 0x7f1ea006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.430+0000 7f1ea56b5640 1 -- 172.21.15.105:0/1222779829 >> 172.21.15.105:0/1222779829 conn(0x7f1ea00d3ce0 msgr2=0x7f1ea00d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.430+0000 7f1ea56b5640 1 -- 172.21.15.105:0/1222779829 shutdown_connections 2024-03-08T08:28:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.430+0000 7f1ea56b5640 1 -- 172.21.15.105:0/1222779829 wait complete. 2024-03-08T08:28:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.431+0000 7f1ea56b5640 1 Processor -- start 2024-03-08T08:28:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.431+0000 7f1ea56b5640 1 -- start start 2024-03-08T08:28:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1ea56b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea006d370 0x7f1ea00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1ea56b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ea0161470 0x7f1ea01f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1ea56b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea01f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1ea56b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea006cf40 con 0x7f1ea0161850 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1ea56b5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea006bcb0 con 0x7f1ea0161470 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1ea56b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00b0d30 con 0x7f1ea006d370 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea01f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea01f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34888/0 (socket says 172.21.15.105:34888) 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 -- 172.21.15.105:0/2386918085 learned_addr learned my addr 172.21.15.105:0/2386918085 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 -- 172.21.15.105:0/2386918085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea006d370 msgr2=0x7f1ea00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea006d370 0x7f1ea00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 -- 172.21.15.105:0/2386918085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ea0161470 msgr2=0x7f1ea01f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ea0161470 0x7f1ea01f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 -- 172.21.15.105:0/2386918085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea01f4870 con 0x7f1ea0161850 2024-03-08T08:28:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e9f7fe640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea01f3ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800c440 tx=0x7f1e9800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e877fe640 1 -- 172.21.15.105:0/2386918085 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98011790 con 0x7f1ea0161850 2024-03-08T08:28:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e877fe640 1 -- 172.21.15.105:0/2386918085 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e98017070 con 0x7f1ea0161850 2024-03-08T08:28:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1e877fe640 1 -- 172.21.15.105:0/2386918085 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98011b70 con 0x7f1ea0161850 2024-03-08T08:28:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea01f4590 con 0x7f1ea0161850 2024-03-08T08:28:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.432+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea00b3120 con 0x7f1ea0161850 2024-03-08T08:28:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.433+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea0075120 con 0x7f1ea0161850 2024-03-08T08:28:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.434+0000 7f1e877fe640 1 -- 172.21.15.105:0/2386918085 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e98011d10 con 0x7f1ea0161850 2024-03-08T08:28:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.434+0000 7f1e877fe640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1e8c06b4d0 0x7f1e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.434+0000 7f1e877fe640 1 -- 172.21.15.105:0/2386918085 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1e98089cc0 con 0x7f1ea0161850 2024-03-08T08:28:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.434+0000 7f1e9effd640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1e8c06b4d0 0x7f1e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.434+0000 7f1e9effd640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1e8c06b4d0 0x7f1e8c06d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00d46a0 tx=0x7f1e900083d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.438+0000 7f1e877fe640 1 -- 172.21.15.105:0/2386918085 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e98058e90 con 0x7f1ea0161850 2024-03-08T08:28:27.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.676+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f1ea00b96c0 con 0x7f1ea0161850 2024-03-08T08:28:27.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.678+0000 7f1e877fe640 1 -- 172.21.15.105:0/2386918085 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1e98058790 con 0x7f1ea0161850 2024-03-08T08:28:27.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1e8c06b4d0 msgr2=0x7f1e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1e8c06b4d0 0x7f1e8c06d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00d46a0 tx=0x7f1e900083d0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 msgr2=0x7f1ea01f3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea01f3ef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800c440 tx=0x7f1e9800c900 comp rx=0 tx=0).stop 2024-03-08T08:28:27.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 shutdown_connections 2024-03-08T08:28:27.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1e8c06b4d0 0x7f1e8c06d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ea0161850 0x7f1ea01f3ef0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ea0161470 0x7f1ea01f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 --2- 172.21.15.105:0/2386918085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ea006d370 0x7f1ea00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 >> 172.21.15.105:0/2386918085 conn(0x7f1ea00d3ce0 msgr2=0x7f1ea006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:27.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 shutdown_connections 2024-03-08T08:28:27.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.680+0000 7f1ea56b5640 1 -- 172.21.15.105:0/2386918085 wait complete. 2024-03-08T08:28:27.693 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-08T08:28:27.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.791+0000 7f1910948640 1 Processor -- start 2024-03-08T08:28:27.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.791+0000 7f1910948640 1 -- start start 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f1910948640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f1910948640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f190c0ce330 0x7f190c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f1910948640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c0b29e0 con 0x7f190c0b14f0 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f1910948640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c0d1930 con 0x7f190c0ce330 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f1910948640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0cecd0 con 0x7f190c161520 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f190a59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f1909d9b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f190c0ce330 0x7f190c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f190a59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:39726/0 (socket says 172.21.15.105:39726) 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.792+0000 7f190a59c640 1 -- 172.21.15.105:0/53606089 learned_addr learned my addr 172.21.15.105:0/53606089 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.793+0000 7f1909d9b640 1 -- 172.21.15.105:0/53606089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 msgr2=0x7f190c0b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.793+0000 7f1909d9b640 1 --2- 172.21.15.105:0/53606089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.793+0000 7f1909d9b640 1 -- 172.21.15.105:0/53606089 >> v1:172.21.15.144:6789/0 conn(0x7f190c0b14f0 legacy=0x7f190c0b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.793+0000 7f1909d9b640 1 -- 172.21.15.105:0/53606089 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c0d1650 con 0x7f190c0ce330 2024-03-08T08:28:27.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1909d9b640 1 --2- 172.21.15.105:0/53606089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 0x7f190c0ce790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f18f40097a0 tx=0x7f18f402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff57063fd75e668d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f190959a640 1 -- 172.21.15.105:0/53606089 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4035020 con 0x7f190c0ce330 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f190959a640 1 -- 172.21.15.105:0/53606089 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f4031830 con 0x7f190c0ce330 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f190959a640 1 -- 172.21.15.105:0/53606089 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4031c00 con 0x7f190c0ce330 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1910948640 1 -- 172.21.15.105:0/53606089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 msgr2=0x7f190c0ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1910948640 1 --2- 172.21.15.105:0/53606089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 0x7f190c0ce790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f18f40097a0 tx=0x7f18f402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1910948640 1 -- 172.21.15.105:0/53606089 shutdown_connections 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1910948640 1 --2- 172.21.15.105:0/53606089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 0x7f190c0ce790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1910948640 1 --2- 172.21.15.105:0/53606089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1910948640 1 -- 172.21.15.105:0/53606089 >> 172.21.15.105:0/53606089 conn(0x7f190c0b36a0 msgr2=0x7f190c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1910948640 1 -- 172.21.15.105:0/53606089 shutdown_connections 2024-03-08T08:28:27.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.794+0000 7f1910948640 1 -- 172.21.15.105:0/53606089 wait complete. 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f1910948640 1 Processor -- start 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f1910948640 1 -- start start 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f1910948640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c0b14f0 0x7f190c0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f1910948640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 0x7f190c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f1910948640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f1910948640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c06ca40 con 0x7f190c0b14f0 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f1910948640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c06c770 con 0x7f190c0ce330 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f1910948640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c0b32d0 con 0x7f190c161520 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f190a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c0b14f0 0x7f190c0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f190a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c0b14f0 0x7f190c0cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34900/0 (socket says 172.21.15.105:34900) 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.796+0000 7f190a59c640 1 -- 172.21.15.105:0/1372778890 learned_addr learned my addr 172.21.15.105:0/1372778890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:27.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f1909d9b640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 0x7f190c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f190a59c640 1 -- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 msgr2=0x7f190c0b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f190ad9d640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f190a59c640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f190a59c640 1 -- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 msgr2=0x7f190c0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f190a59c640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 0x7f190c0b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f190a59c640 1 -- 172.21.15.105:0/1372778890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c0b4db0 con 0x7f190c0b14f0 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f1909d9b640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 0x7f190c0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f190a59c640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c0b14f0 0x7f190c0cb2b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f190400efc0 tx=0x7f190400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f1902ffd640 1 -- 172.21.15.105:0/1372778890 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400cda0 con 0x7f190c0b14f0 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f1902ffd640 1 -- 172.21.15.105:0/1372778890 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904019070 con 0x7f190c0b14f0 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c0b5090 con 0x7f190c0b14f0 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f1902ffd640 1 -- 172.21.15.105:0/1372778890 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040146a0 con 0x7f190c0b14f0 2024-03-08T08:28:27.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.797+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c0b5930 con 0x7f190c0b14f0 2024-03-08T08:28:27.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.798+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c075120 con 0x7f190c0b14f0 2024-03-08T08:28:27.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.798+0000 7f1902ffd640 1 -- 172.21.15.105:0/1372778890 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1904005d10 con 0x7f190c0b14f0 2024-03-08T08:28:27.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.799+0000 7f1902ffd640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f18ec06b4d0 0x7f18ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:27.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.799+0000 7f1902ffd640 1 -- 172.21.15.105:0/1372778890 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f190408a380 con 0x7f190c0b14f0 2024-03-08T08:28:27.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.799+0000 7f1909d9b640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f18ec06b4d0 0x7f18ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:27.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.799+0000 7f1909d9b640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f18ec06b4d0 0x7f18ec06d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18f4036000 tx=0x7f18f4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:27.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:27.803+0000 7f1902ffd640 1 -- 172.21.15.105:0/1372778890 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1904010040 con 0x7f190c0b14f0 2024-03-08T08:28:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.043+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f190c0ce8c0 con 0x7f190c0b14f0 2024-03-08T08:28:28.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.045+0000 7f1902ffd640 1 -- 172.21.15.105:0/1372778890 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1904059550 con 0x7f190c0b14f0 2024-03-08T08:28:28.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f18ec06b4d0 msgr2=0x7f18ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f18ec06b4d0 0x7f18ec06d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18f4036000 tx=0x7f18f4008030 comp rx=0 tx=0).stop 2024-03-08T08:28:28.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c0b14f0 msgr2=0x7f190c0cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c0b14f0 0x7f190c0cb2b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f190400efc0 tx=0x7f190400c490 comp rx=0 tx=0).stop 2024-03-08T08:28:28.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 shutdown_connections 2024-03-08T08:28:28.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f18ec06b4d0 0x7f18ec06d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f190c161520 0x7f190c0b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f190c0ce330 0x7f190c0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 --2- 172.21.15.105:0/1372778890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f190c0b14f0 0x7f190c0cb2b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 >> 172.21.15.105:0/1372778890 conn(0x7f190c0b36a0 msgr2=0x7f190c0d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:28.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 shutdown_connections 2024-03-08T08:28:28.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.047+0000 7f1910948640 1 -- 172.21.15.105:0/1372778890 wait complete. 2024-03-08T08:28:28.060 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-08T08:28:28.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.158+0000 7fbe20719640 1 Processor -- start 2024-03-08T08:28:28.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.158+0000 7fbe20719640 1 -- start start 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe20719640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe180b4000 0x7fbe1815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe20719640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe180cf290 0x7fbe180b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe20719640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1815b700 con 0x7fbe1806cf90 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe20719640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe180cd9e0 con 0x7fbe180b4000 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe20719640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe180b4920 con 0x7fbe180cf290 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe1e4b5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe180b4000 0x7fbe1815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe1ecb6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbe1806cf90 0x7fbe180cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39198/0 (socket says 172.21.15.105:39198) 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe1ecb6640 1 -- 172.21.15.105:0/1863583086 learned_addr learned my addr 172.21.15.105:0/1863583086 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.159+0000 7fbe1dcb4640 1 --2- 172.21.15.105:0/1863583086 >> v2:172.21.15.144:3300/0 conn(0x7fbe180cf290 0x7fbe180b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1e4b5640 1 -- 172.21.15.105:0/1863583086 >> v2:172.21.15.144:3300/0 conn(0x7fbe180cf290 msgr2=0x7fbe180b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1e4b5640 1 --2- 172.21.15.105:0/1863583086 >> v2:172.21.15.144:3300/0 conn(0x7fbe180cf290 0x7fbe180b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1e4b5640 1 -- 172.21.15.105:0/1863583086 >> v1:172.21.15.105:6789/0 conn(0x7fbe1806cf90 legacy=0x7fbe180cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1e4b5640 1 -- 172.21.15.105:0/1863583086 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18072120 con 0x7fbe180b4000 2024-03-08T08:28:28.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1d4b3640 1 -- 172.21.15.105:0/1863583086 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614281850 0 0) 0x7fbe10001110 con 0x7fbe1806cf90 2024-03-08T08:28:28.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1e4b5640 1 --2- 172.21.15.105:0/1863583086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe180b4000 0x7fbe1815b1c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe180b3b20 tx=0x7fbe1402cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ac545c90fc75ab server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1d4b3640 1 -- 172.21.15.105:0/1863583086 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1402db70 con 0x7fbe180b4000 2024-03-08T08:28:28.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1d4b3640 1 -- 172.21.15.105:0/1863583086 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1402dd10 con 0x7fbe180b4000 2024-03-08T08:28:28.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.160+0000 7fbe1d4b3640 1 -- 172.21.15.105:0/1863583086 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14008660 con 0x7fbe180b4000 2024-03-08T08:28:28.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.161+0000 7fbe20719640 1 -- 172.21.15.105:0/1863583086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe180b4000 msgr2=0x7fbe1815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.161+0000 7fbe20719640 1 --2- 172.21.15.105:0/1863583086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe180b4000 0x7fbe1815b1c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe180b3b20 tx=0x7fbe1402cfd0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.161+0000 7fbe20719640 1 -- 172.21.15.105:0/1863583086 shutdown_connections 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.161+0000 7fbe20719640 1 --2- 172.21.15.105:0/1863583086 >> v2:172.21.15.144:3300/0 conn(0x7fbe180cf290 0x7fbe180b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.161+0000 7fbe20719640 1 --2- 172.21.15.105:0/1863583086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe180b4000 0x7fbe1815b1c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.161+0000 7fbe20719640 1 -- 172.21.15.105:0/1863583086 >> 172.21.15.105:0/1863583086 conn(0x7fbe180b1740 msgr2=0x7fbe180b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.161+0000 7fbe20719640 1 -- 172.21.15.105:0/1863583086 shutdown_connections 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.161+0000 7fbe20719640 1 -- 172.21.15.105:0/1863583086 wait complete. 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.162+0000 7fbe20719640 1 Processor -- start 2024-03-08T08:28:28.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.162+0000 7fbe20719640 1 -- start start 2024-03-08T08:28:28.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe20719640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1806cf90 0x7fbe18155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe20719640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe180b4000 0x7fbe181537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe20719640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe180cf290 0x7fbe181f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe20719640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180726e0 con 0x7fbe1806cf90 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe20719640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180cd670 con 0x7fbe180cf290 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe20719640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1815a500 con 0x7fbe180b4000 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1806cf90 0x7fbe18155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1806cf90 0x7fbe18155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34928/0 (socket says 172.21.15.105:34928) 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 -- 172.21.15.105:0/3865823796 learned_addr learned my addr 172.21.15.105:0/3865823796 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 -- 172.21.15.105:0/3865823796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe180b4000 msgr2=0x7fbe181537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe180b4000 0x7fbe181537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 -- 172.21.15.105:0/3865823796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe180cf290 msgr2=0x7fbe181f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe180cf290 0x7fbe181f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 -- 172.21.15.105:0/3865823796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18142f50 con 0x7fbe1806cf90 2024-03-08T08:28:28.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.163+0000 7fbe1e4b5640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1806cf90 0x7fbe18155f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe14000f80 tx=0x7fbe14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.164+0000 7fbe0effd640 1 -- 172.21.15.105:0/3865823796 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14044020 con 0x7fbe1806cf90 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.164+0000 7fbe0effd640 1 -- 172.21.15.105:0/3865823796 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14040670 con 0x7fbe1806cf90 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.164+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe181f5ad0 con 0x7fbe1806cf90 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.164+0000 7fbe0effd640 1 -- 172.21.15.105:0/3865823796 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400a560 con 0x7fbe1806cf90 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.164+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe18155460 con 0x7fbe1806cf90 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.164+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe18075120 con 0x7fbe1806cf90 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.165+0000 7fbe0effd640 1 -- 172.21.15.105:0/3865823796 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe14047020 con 0x7fbe1806cf90 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.165+0000 7fbe0effd640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbdfc06b4d0 0x7fbdfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.165+0000 7fbe0effd640 1 -- 172.21.15.105:0/3865823796 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbe140acd20 con 0x7fbe1806cf90 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.165+0000 7fbe1dcb4640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbdfc06b4d0 0x7fbdfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.166+0000 7fbe1dcb4640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbdfc06b4d0 0x7fbdfc06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe1806e5e0 tx=0x7fbe08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.170+0000 7fbe0effd640 1 -- 172.21.15.105:0/3865823796 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe1404e030 con 0x7fbe1806cf90 2024-03-08T08:28:28.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.410+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fbe181569f0 con 0x7fbe1806cf90 2024-03-08T08:28:28.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.412+0000 7fbe0effd640 1 -- 172.21.15.105:0/3865823796 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbe1407c010 con 0x7fbe1806cf90 2024-03-08T08:28:28.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbdfc06b4d0 msgr2=0x7fbdfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbdfc06b4d0 0x7fbdfc06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe1806e5e0 tx=0x7fbe08008040 comp rx=0 tx=0).stop 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1806cf90 msgr2=0x7fbe18155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1806cf90 0x7fbe18155f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe14000f80 tx=0x7fbe14004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 shutdown_connections 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbdfc06b4d0 0x7fbdfc06d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe180cf290 0x7fbe181f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe180b4000 0x7fbe181537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 --2- 172.21.15.105:0/3865823796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe1806cf90 0x7fbe18155f40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 >> 172.21.15.105:0/3865823796 conn(0x7fbe180b1740 msgr2=0x7fbe181571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 shutdown_connections 2024-03-08T08:28:28.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.414+0000 7fbe20719640 1 -- 172.21.15.105:0/3865823796 wait complete. 2024-03-08T08:28:28.427 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:28.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.527+0000 7fb36a5b5640 1 Processor -- start 2024-03-08T08:28:28.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.527+0000 7fb36a5b5640 1 -- start start 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb36a5b5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb36415e340 0x7fb36406b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb36a5b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb364066990 0x7fb3640d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb36a5b5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36415e720 con 0x7fb3640d05f0 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb36a5b5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36415f020 con 0x7fb36415e340 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb36a5b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb364066ec0 con 0x7fb364066990 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb363fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb36415e340 0x7fb36406b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb368b52640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb3640d05f0 0x7fb364161c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39202/0 (socket says 172.21.15.105:39202) 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb368b52640 1 -- 172.21.15.105:0/431354177 learned_addr learned my addr 172.21.15.105:0/431354177 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb363fff640 1 -- 172.21.15.105:0/431354177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb364066990 msgr2=0x7fb3640d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb363fff640 1 --2- 172.21.15.105:0/431354177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb364066990 0x7fb3640d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb363fff640 1 -- 172.21.15.105:0/431354177 >> v1:172.21.15.105:6789/0 conn(0x7fb3640d05f0 legacy=0x7fb364161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.528+0000 7fb363fff640 1 -- 172.21.15.105:0/431354177 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36415e9d0 con 0x7fb36415e340 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.529+0000 7fb363fff640 1 --2- 172.21.15.105:0/431354177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36415e340 0x7fb36406b690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097d0 tx=0x7fb35802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e446186ddadff60 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.529+0000 7fb362ffd640 1 -- 172.21.15.105:0/431354177 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358039070 con 0x7fb36415e340 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.529+0000 7fb362ffd640 1 -- 172.21.15.105:0/431354177 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3580316f0 con 0x7fb36415e340 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.529+0000 7fb362ffd640 1 -- 172.21.15.105:0/431354177 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358031ac0 con 0x7fb36415e340 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.530+0000 7fb36a5b5640 1 -- 172.21.15.105:0/431354177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36415e340 msgr2=0x7fb36406b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.530+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/431354177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36415e340 0x7fb36406b690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097d0 tx=0x7fb35802b790 comp rx=0 tx=0).stop 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.530+0000 7fb36a5b5640 1 -- 172.21.15.105:0/431354177 shutdown_connections 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.530+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/431354177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb364066990 0x7fb3640d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.530+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/431354177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb36415e340 0x7fb36406b690 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.530+0000 7fb36a5b5640 1 -- 172.21.15.105:0/431354177 >> 172.21.15.105:0/431354177 conn(0x7fb364072110 msgr2=0x7fb364072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.530+0000 7fb36a5b5640 1 -- 172.21.15.105:0/431354177 shutdown_connections 2024-03-08T08:28:28.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.530+0000 7fb36a5b5640 1 -- 172.21.15.105:0/431354177 wait complete. 2024-03-08T08:28:28.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.531+0000 7fb36a5b5640 1 Processor -- start 2024-03-08T08:28:28.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.531+0000 7fb36a5b5640 1 -- start start 2024-03-08T08:28:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb36a5b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb364066990 0x7fb364157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb36a5b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3640d05f0 0x7fb36414cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb36a5b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb36415e340 0x7fb3641f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb36a5b5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415f4b0 con 0x7fb3640d05f0 2024-03-08T08:28:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb36a5b5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640af530 con 0x7fb364066990 2024-03-08T08:28:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb36a5b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb364161c10 con 0x7fb36415e340 2024-03-08T08:28:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3640d05f0 0x7fb36414cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3640d05f0 0x7fb36414cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34936/0 (socket says 172.21.15.105:34936) 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 -- 172.21.15.105:0/770144456 learned_addr learned my addr 172.21.15.105:0/770144456 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 -- 172.21.15.105:0/770144456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb36415e340 msgr2=0x7fb3641f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb36415e340 0x7fb3641f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 -- 172.21.15.105:0/770144456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb364066990 msgr2=0x7fb364157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb364066990 0x7fb364157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 -- 172.21.15.105:0/770144456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3641612e0 con 0x7fb3640d05f0 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb3637fe640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3640d05f0 0x7fb36414cde0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb35400b410 tx=0x7fb35400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb360ff9640 1 -- 172.21.15.105:0/770144456 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354015020 con 0x7fb3640d05f0 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb360ff9640 1 -- 172.21.15.105:0/770144456 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb354011830 con 0x7fb3640d05f0 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3641615c0 con 0x7fb3640d05f0 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb360ff9640 1 -- 172.21.15.105:0/770144456 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354011c20 con 0x7fb3640d05f0 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.532+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36415ab90 con 0x7fb3640d05f0 2024-03-08T08:28:28.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.534+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb364161790 con 0x7fb3640d05f0 2024-03-08T08:28:28.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.534+0000 7fb360ff9640 1 -- 172.21.15.105:0/770144456 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb354018020 con 0x7fb3640d05f0 2024-03-08T08:28:28.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.534+0000 7fb360ff9640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35006b4d0 0x7fb35006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.534+0000 7fb360ff9640 1 -- 172.21.15.105:0/770144456 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb354089de0 con 0x7fb3640d05f0 2024-03-08T08:28:28.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.534+0000 7fb363fff640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35006b4d0 0x7fb35006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.535+0000 7fb363fff640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35006b4d0 0x7fb35006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb358036000 tx=0x7fb358030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.538+0000 7fb360ff9640 1 -- 172.21.15.105:0/770144456 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb35401f020 con 0x7fb3640d05f0 2024-03-08T08:28:28.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.719+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3641619b0 con 0x7fb3640d05f0 2024-03-08T08:28:28.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.719+0000 7fb360ff9640 1 -- 172.21.15.105:0/770144456 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12903 (secure 0 0 0) 0x7fb3540172c0 con 0x7fb3640d05f0 2024-03-08T08:28:28.721 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:28.722 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":29,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:25.862177+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35006b4d0 msgr2=0x7fb35006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35006b4d0 0x7fb35006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb358036000 tx=0x7fb358030000 comp rx=0 tx=0).stop 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3640d05f0 msgr2=0x7fb36414cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3640d05f0 0x7fb36414cde0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb35400b410 tx=0x7fb35400b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 shutdown_connections 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35006b4d0 0x7fb35006d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb36415e340 0x7fb3641f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3640d05f0 0x7fb36414cde0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 --2- 172.21.15.105:0/770144456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb364066990 0x7fb364157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 >> 172.21.15.105:0/770144456 conn(0x7fb364072110 msgr2=0x7fb3640b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:28.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 shutdown_connections 2024-03-08T08:28:28.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.722+0000 7fb36a5b5640 1 -- 172.21.15.105:0/770144456 wait complete. 2024-03-08T08:28:28.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:28:28.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:28:28.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.843+0000 7f5546b81640 1 Processor -- start 2024-03-08T08:28:28.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.843+0000 7f5546b81640 1 -- start start 2024-03-08T08:28:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.843+0000 7f5546b81640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f554015aad0 0x7f554015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.843+0000 7f5546b81640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f554015aeb0 0x7f554006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.843+0000 7f5546b81640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554015b360 con 0x7f554006ec20 2024-03-08T08:28:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.843+0000 7f5546b81640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554006f070 con 0x7f554015aad0 2024-03-08T08:28:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.843+0000 7f5546b81640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554015ebb0 con 0x7f554015aeb0 2024-03-08T08:28:28.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f554491d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f554015aad0 0x7f554015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f554491d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f554015aad0 0x7f554015e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34946/0 (socket says 172.21.15.105:34946) 2024-03-08T08:28:28.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f554491d640 1 -- 172.21.15.105:0/174513310 learned_addr learned my addr 172.21.15.105:0/174513310 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:28.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f553ffff640 1 --2- 172.21.15.105:0/174513310 >> v2:172.21.15.144:3300/0 conn(0x7f554015aeb0 0x7f554006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f554491d640 1 -- 172.21.15.105:0/174513310 >> v2:172.21.15.144:3300/0 conn(0x7f554015aeb0 msgr2=0x7f554006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:28.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f554491d640 1 --2- 172.21.15.105:0/174513310 >> v2:172.21.15.144:3300/0 conn(0x7f554015aeb0 0x7f554006e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f554491d640 1 -- 172.21.15.105:0/174513310 >> v1:172.21.15.144:6789/0 conn(0x7f554006ec20 legacy=0x7f554006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f554491d640 1 -- 172.21.15.105:0/174513310 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554015ee90 con 0x7f554015aad0 2024-03-08T08:28:28.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.844+0000 7f553ffff640 1 --2- 172.21.15.105:0/174513310 >> v2:172.21.15.144:3300/0 conn(0x7f554015aeb0 0x7f554006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:28.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.845+0000 7f554491d640 1 --2- 172.21.15.105:0/174513310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554015aad0 0x7f554015e5d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5540073370 tx=0x7f553802e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8acb2cea16e1566 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.845+0000 7f553f7fe640 1 -- 172.21.15.105:0/174513310 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553800b0d0 con 0x7f554015aad0 2024-03-08T08:28:28.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.845+0000 7f553f7fe640 1 -- 172.21.15.105:0/174513310 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553800b270 con 0x7f554015aad0 2024-03-08T08:28:28.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.845+0000 7f553f7fe640 1 -- 172.21.15.105:0/174513310 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538009600 con 0x7f554015aad0 2024-03-08T08:28:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.846+0000 7f5546b81640 1 -- 172.21.15.105:0/174513310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554015aad0 msgr2=0x7f554015e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.846+0000 7f5546b81640 1 --2- 172.21.15.105:0/174513310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554015aad0 0x7f554015e5d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5540073370 tx=0x7f553802e180 comp rx=0 tx=0).stop 2024-03-08T08:28:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.846+0000 7f5546b81640 1 -- 172.21.15.105:0/174513310 shutdown_connections 2024-03-08T08:28:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.846+0000 7f5546b81640 1 --2- 172.21.15.105:0/174513310 >> v2:172.21.15.144:3300/0 conn(0x7f554015aeb0 0x7f554006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.846+0000 7f5546b81640 1 --2- 172.21.15.105:0/174513310 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554015aad0 0x7f554015e5d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.846+0000 7f5546b81640 1 -- 172.21.15.105:0/174513310 >> 172.21.15.105:0/174513310 conn(0x7f554006bd90 msgr2=0x7f554006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.846+0000 7f5546b81640 1 -- 172.21.15.105:0/174513310 shutdown_connections 2024-03-08T08:28:28.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.846+0000 7f5546b81640 1 -- 172.21.15.105:0/174513310 wait complete. 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.847+0000 7f5546b81640 1 Processor -- start 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.847+0000 7f5546b81640 1 -- start start 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f5546b81640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554006ec20 0x7f5540159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f5546b81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f554015aad0 0x7f55401492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f5546b81640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f554015aeb0 0x7f55400a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f5546b81640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554015c240 con 0x7f554006ec20 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f5546b81640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554015f450 con 0x7f554015aad0 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f5546b81640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55400728a0 con 0x7f554015aeb0 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554491d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554006ec20 0x7f5540159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554491d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554006ec20 0x7f5540159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34950/0 (socket says 172.21.15.105:34950) 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554491d640 1 -- 172.21.15.105:0/2735504391 learned_addr learned my addr 172.21.15.105:0/2735504391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554511e640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f554015aeb0 0x7f55400a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f553ffff640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f554015aad0 0x7f55401492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554491d640 1 -- 172.21.15.105:0/2735504391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f554015aeb0 msgr2=0x7f55400a9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554491d640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f554015aeb0 0x7f55400a9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554491d640 1 -- 172.21.15.105:0/2735504391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f554015aad0 msgr2=0x7f55401492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554491d640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f554015aad0 0x7f55401492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.848+0000 7f554491d640 1 -- 172.21.15.105:0/2735504391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55401582d0 con 0x7f554006ec20 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.849+0000 7f554491d640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554006ec20 0x7f5540159370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5538000f80 tx=0x7f5538004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.850+0000 7f553d7fa640 1 -- 172.21.15.105:0/2735504391 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538045020 con 0x7f554006ec20 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.850+0000 7f553d7fa640 1 -- 172.21.15.105:0/2735504391 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553800b420 con 0x7f554006ec20 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.850+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55401585b0 con 0x7f554006ec20 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.850+0000 7f553d7fa640 1 -- 172.21.15.105:0/2735504391 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553800b0d0 con 0x7f554006ec20 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.850+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5540158e50 con 0x7f554006ec20 2024-03-08T08:28:28.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.851+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55401589a0 con 0x7f554006ec20 2024-03-08T08:28:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.852+0000 7f553d7fa640 1 -- 172.21.15.105:0/2735504391 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5538048020 con 0x7f554006ec20 2024-03-08T08:28:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.852+0000 7f553d7fa640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f552c06b4d0 0x7f552c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.852+0000 7f553d7fa640 1 -- 172.21.15.105:0/2735504391 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f55380ae8d0 con 0x7f554006ec20 2024-03-08T08:28:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.852+0000 7f553ffff640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f552c06b4d0 0x7f552c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:28.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.853+0000 7f553ffff640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f552c06b4d0 0x7f552c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f55400b0510 tx=0x7f5534008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:28.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:28.857+0000 7f553d7fa640 1 -- 172.21.15.105:0/2735504391 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f553804f030 con 0x7f554006ec20 2024-03-08T08:28:29.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.032+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5540075120 con 0x7f554006ec20 2024-03-08T08:28:29.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.032+0000 7f553d7fa640 1 -- 172.21.15.105:0/2735504391 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f553807d570 con 0x7f554006ec20 2024-03-08T08:28:29.034 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:28:29.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f552c06b4d0 msgr2=0x7f552c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f552c06b4d0 0x7f552c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f55400b0510 tx=0x7f5534008040 comp rx=0 tx=0).stop 2024-03-08T08:28:29.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554006ec20 msgr2=0x7f5540159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554006ec20 0x7f5540159370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5538000f80 tx=0x7f5538004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 shutdown_connections 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f552c06b4d0 0x7f552c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f554015aeb0 0x7f55400a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f554015aad0 0x7f55401492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 --2- 172.21.15.105:0/2735504391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f554006ec20 0x7f5540159370 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 >> 172.21.15.105:0/2735504391 conn(0x7f554006bd90 msgr2=0x7f55400b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 shutdown_connections 2024-03-08T08:28:29.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.034+0000 7f5546b81640 1 -- 172.21.15.105:0/2735504391 wait complete. 2024-03-08T08:28:29.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-08T08:28:29.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.161+0000 7f410326b640 1 Processor -- start 2024-03-08T08:28:29.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.161+0000 7f410326b640 1 -- start start 2024-03-08T08:28:29.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f410326b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f40fc15df20 0x7f40fc06c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f410326b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 0x7f40fc0ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f410326b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc06f830 con 0x7f40fc0b5990 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f410326b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc06dbb0 con 0x7f40fc15df20 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f410326b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06fad0 con 0x7f40fc06b390 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f4100806640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f40fc0b5990 0x7f40fc06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39208/0 (socket says 172.21.15.105:39208) 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f4100806640 1 -- 172.21.15.105:0/1838416708 learned_addr learned my addr 172.21.15.105:0/1838416708 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f4101007640 1 --2- 172.21.15.105:0/1838416708 >> v2:172.21.15.144:3300/0 conn(0x7f40fc15df20 0x7f40fc06c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f4101808640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 0x7f40fc0ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488211142 0 0) 0x7f40fc06f830 con 0x7f40fc0b5990 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e8003520 con 0x7f40fc0b5990 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1236744258 0 0) 0x7f40e8003520 con 0x7f40fc0b5990 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.162+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 msgr2=0x7f40fc0ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 0x7f40fc0ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 >> v2:172.21.15.144:3300/0 conn(0x7f40fc15df20 msgr2=0x7f40fc06c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 --2- 172.21.15.105:0/1838416708 >> v2:172.21.15.144:3300/0 conn(0x7f40fc15df20 0x7f40fc06c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc06fdb0 con 0x7f40fc0b5990 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f40f8001690 con 0x7f40fc0b5990 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101007640 1 --2- 172.21.15.105:0/1838416708 >> v2:172.21.15.144:3300/0 conn(0x7f40fc15df20 0x7f40fc06c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 >> v1:172.21.15.105:6789/0 conn(0x7f40fc0b5990 legacy=0x7f40fc06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 0x7f40e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40e8004d40 0x7f40e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06dbb0 con 0x7f40e80039b0 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06fad0 con 0x7f40e8004d40 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e800b9f0 con 0x7f40fc06b390 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101007640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4100806640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 0x7f40e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101808640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40e8004d40 0x7f40e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101007640 1 -- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 msgr2=0x7f40e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101007640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 0x7f40e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101007640 1 -- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40e8004d40 msgr2=0x7f40e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101007640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40e8004d40 0x7f40e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101007640 1 -- 172.21.15.105:0/1838416708 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8003520 con 0x7f40e80039b0 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4100806640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc06b390 0x7f40e8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.163+0000 7f4101808640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40e8004d40 0x7f40e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f4101007640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40e8003dd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009c90 tx=0x7f40e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342102c175f48934 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec039070 con 0x7f40e80039b0 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec02c980 con 0x7f40e80039b0 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f40f3fff640 1 -- 172.21.15.105:0/1838416708 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec009860 con 0x7f40e80039b0 2024-03-08T08:28:29.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f410326b640 1 -- 172.21.15.105:0/1838416708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 msgr2=0x7f40e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f410326b640 1 --2- 172.21.15.105:0/1838416708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40e8003dd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009c90 tx=0x7f40e8005830 comp rx=0 tx=0).stop 2024-03-08T08:28:29.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f4101808640 1 -- 172.21.15.105:0/1838416708 reap_dead start 2024-03-08T08:28:29.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f410326b640 1 -- 172.21.15.105:0/1838416708 shutdown_connections 2024-03-08T08:28:29.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f410326b640 1 -- 172.21.15.105:0/1838416708 >> 172.21.15.105:0/1838416708 conn(0x7f40fc06bd90 msgr2=0x7f40fc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:29.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f410326b640 1 -- 172.21.15.105:0/1838416708 shutdown_connections 2024-03-08T08:28:29.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.164+0000 7f410326b640 1 -- 172.21.15.105:0/1838416708 wait complete. 2024-03-08T08:28:29.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f410326b640 1 Processor -- start 2024-03-08T08:28:29.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f410326b640 1 -- start start 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f410326b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40fc13fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f410326b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc156e30 0x7f40fc142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f410326b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40fc154ef0 0x7f40fc155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f410326b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0ad970 con 0x7f40e80039b0 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f410326b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06e0e0 con 0x7f40fc154ef0 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f410326b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0adaf0 con 0x7f40fc156e30 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f4101007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40fc13fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f4101007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40fc13fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34970/0 (socket says 172.21.15.105:34970) 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.166+0000 7f4101007640 1 -- 172.21.15.105:0/3051615282 learned_addr learned my addr 172.21.15.105:0/3051615282 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f4101808640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40fc154ef0 0x7f40fc155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f4100806640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc156e30 0x7f40fc142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f4101007640 1 -- 172.21.15.105:0/3051615282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc156e30 msgr2=0x7f40fc142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f4101007640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc156e30 0x7f40fc142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f4101007640 1 -- 172.21.15.105:0/3051615282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40fc154ef0 msgr2=0x7f40fc155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f4101007640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40fc154ef0 0x7f40fc155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f4101007640 1 -- 172.21.15.105:0/3051615282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc143420 con 0x7f40e80039b0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f4101007640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40fc13fd60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0093d0 tx=0x7f40ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f40f1ffb640 1 -- 172.21.15.105:0/3051615282 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec039050 con 0x7f40e80039b0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f40f1ffb640 1 -- 172.21.15.105:0/3051615282 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec045070 con 0x7f40e80039b0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc157640 con 0x7f40e80039b0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f40f1ffb640 1 -- 172.21.15.105:0/3051615282 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec009860 con 0x7f40e80039b0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.167+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc157c00 con 0x7f40e80039b0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.168+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc075120 con 0x7f40e80039b0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.169+0000 7f40f1ffb640 1 -- 172.21.15.105:0/3051615282 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40ec04a460 con 0x7f40e80039b0 2024-03-08T08:28:29.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.169+0000 7f40f1ffb640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f40e006b4d0 0x7f40e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.169+0000 7f40f1ffb640 1 -- 172.21.15.105:0/3051615282 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40ec0ac1f0 con 0x7f40e80039b0 2024-03-08T08:28:29.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.169+0000 7f4100806640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f40e006b4d0 0x7f40e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.170+0000 7f4100806640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f40e006b4d0 0x7f40e006d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f40f8001eb0 tx=0x7f40f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.177+0000 7f40f1ffb640 1 -- 172.21.15.105:0/3051615282 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40ec07b3c0 con 0x7f40e80039b0 2024-03-08T08:28:29.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.352+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f40fc158470 con 0x7f40e80039b0 2024-03-08T08:28:29.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.352+0000 7f40f1ffb640 1 -- 172.21.15.105:0/3051615282 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f40ec0810c0 con 0x7f40e80039b0 2024-03-08T08:28:29.354 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:29.354 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-03-08T08:28:29.355 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:29.355 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-03-08T08:28:29.355 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-03-08T08:28:29.355 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:29.355 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-03-08T08:28:29.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f40e006b4d0 msgr2=0x7f40e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f40e006b4d0 0x7f40e006d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f40f8001eb0 tx=0x7f40f8009040 comp rx=0 tx=0).stop 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 msgr2=0x7f40fc13fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40fc13fd60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0093d0 tx=0x7f40ec004730 comp rx=0 tx=0).stop 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 shutdown_connections 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f40e006b4d0 0x7f40e006d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40fc154ef0 0x7f40fc155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40fc156e30 0x7f40fc142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 --2- 172.21.15.105:0/3051615282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40e80039b0 0x7f40fc13fd60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 >> 172.21.15.105:0/3051615282 conn(0x7f40fc06bd90 msgr2=0x7f40fc06ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 shutdown_connections 2024-03-08T08:28:29.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.354+0000 7f410326b640 1 -- 172.21.15.105:0/3051615282 wait complete. 2024-03-08T08:28:29.367 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-08T08:28:29.367 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-08T08:28:29.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.469+0000 7fe23db0d640 1 Processor -- start 2024-03-08T08:28:29.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.470+0000 7fe23db0d640 1 -- start start 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.470+0000 7fe23db0d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe238140e70 0x7fe2380b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.470+0000 7fe23db0d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe238141250 con 0x7fe238144200 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.470+0000 7fe23db0d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe238143740 con 0x7fe23815eb30 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.470+0000 7fe23db0d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23806b750 con 0x7fe238140e70 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.470+0000 7fe2377fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe238140e70 0x7fe2380b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.470+0000 7fe236ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe238144200 0x7fe23815e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39216/0 (socket says 172.21.15.105:39216) 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.470+0000 7fe236ffd640 1 -- 172.21.15.105:0/323288974 learned_addr learned my addr 172.21.15.105:0/323288974 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.471+0000 7fe2377fe640 1 -- 172.21.15.105:0/323288974 >> v1:172.21.15.144:6789/0 conn(0x7fe23815eb30 legacy=0x7fe2380af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.471+0000 7fe2377fe640 1 -- 172.21.15.105:0/323288974 >> v1:172.21.15.105:6789/0 conn(0x7fe238144200 legacy=0x7fe23815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.471+0000 7fe2377fe640 1 -- 172.21.15.105:0/323288974 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23815f110 con 0x7fe238140e70 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe2377fe640 1 --2- 172.21.15.105:0/323288974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238140e70 0x7fe2380b3b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0097a0 tx=0x7fe22c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d28c7a08bf2e02b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe2367fc640 1 -- 172.21.15.105:0/323288974 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c035020 con 0x7fe238140e70 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe2367fc640 1 -- 172.21.15.105:0/323288974 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c031830 con 0x7fe238140e70 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe2367fc640 1 -- 172.21.15.105:0/323288974 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c031c00 con 0x7fe238140e70 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe23db0d640 1 -- 172.21.15.105:0/323288974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238140e70 msgr2=0x7fe2380b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe23db0d640 1 --2- 172.21.15.105:0/323288974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238140e70 0x7fe2380b3b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0097a0 tx=0x7fe22c02b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe23db0d640 1 -- 172.21.15.105:0/323288974 shutdown_connections 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe23db0d640 1 --2- 172.21.15.105:0/323288974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238140e70 0x7fe2380b3b50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe23db0d640 1 -- 172.21.15.105:0/323288974 >> 172.21.15.105:0/323288974 conn(0x7fe2380d0ae0 msgr2=0x7fe2380d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:29.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe23db0d640 1 -- 172.21.15.105:0/323288974 shutdown_connections 2024-03-08T08:28:29.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.472+0000 7fe23db0d640 1 -- 172.21.15.105:0/323288974 wait complete. 2024-03-08T08:28:29.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.473+0000 7fe23db0d640 1 Processor -- start 2024-03-08T08:28:29.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.474+0000 7fe23db0d640 1 -- start start 2024-03-08T08:28:29.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.474+0000 7fe23db0d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe238140e70 0x7fe238156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.474+0000 7fe23db0d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238144200 0x7fe238155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.474+0000 7fe23db0d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23815eb30 0x7fe238157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.474+0000 7fe23db0d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b4090 con 0x7fe238144200 2024-03-08T08:28:29.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.474+0000 7fe23db0d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe238143d70 con 0x7fe238140e70 2024-03-08T08:28:29.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.474+0000 7fe23db0d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b4210 con 0x7fe23815eb30 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.474+0000 7fe236ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238144200 0x7fe238155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe236ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238144200 0x7fe238155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34986/0 (socket says 172.21.15.105:34986) 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe236ffd640 1 -- 172.21.15.105:0/2350540588 learned_addr learned my addr 172.21.15.105:0/2350540588 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe2377fe640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe238140e70 0x7fe238156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe237fff640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23815eb30 0x7fe238157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe236ffd640 1 -- 172.21.15.105:0/2350540588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23815eb30 msgr2=0x7fe238157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe236ffd640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23815eb30 0x7fe238157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe236ffd640 1 -- 172.21.15.105:0/2350540588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe238140e70 msgr2=0x7fe238156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe236ffd640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe238140e70 0x7fe238156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe236ffd640 1 -- 172.21.15.105:0/2350540588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe238155760 con 0x7fe238144200 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe236ffd640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238144200 0x7fe238155220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c400 tx=0x7fe22800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe21ffff640 1 -- 172.21.15.105:0/2350540588 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228017070 con 0x7fe238144200 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe21ffff640 1 -- 172.21.15.105:0/2350540588 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe228005ce0 con 0x7fe238144200 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe238155a00 con 0x7fe238144200 2024-03-08T08:28:29.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe21ffff640 1 -- 172.21.15.105:0/2350540588 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2280072f0 con 0x7fe238144200 2024-03-08T08:28:29.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.475+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe238155fc0 con 0x7fe238144200 2024-03-08T08:28:29.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.476+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe238075200 con 0x7fe238144200 2024-03-08T08:28:29.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.477+0000 7fe21ffff640 1 -- 172.21.15.105:0/2350540588 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe22801a070 con 0x7fe238144200 2024-03-08T08:28:29.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.477+0000 7fe21ffff640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22406b4d0 0x7fe22406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.477+0000 7fe21ffff640 1 -- 172.21.15.105:0/2350540588 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe2280892f0 con 0x7fe238144200 2024-03-08T08:28:29.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.477+0000 7fe2377fe640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22406b4d0 0x7fe22406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.478+0000 7fe2377fe640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22406b4d0 0x7fe22406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe22c036000 tx=0x7fe22c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.481+0000 7fe21ffff640 1 -- 172.21.15.105:0/2350540588 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe238141250 con 0x7fe238144200 2024-03-08T08:28:29.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.655+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe23806b8f0 con 0x7fe238144200 2024-03-08T08:28:29.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.852+0000 7fe21ffff640 1 -- 172.21.15.105:0/2350540588 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe2280584c0 con 0x7fe238144200 2024-03-08T08:28:29.854 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.854+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22406b4d0 msgr2=0x7fe22406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.854+0000 7fe23db0d640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22406b4d0 0x7fe22406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe22c036000 tx=0x7fe22c008030 comp rx=0 tx=0).stop 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.854+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238144200 msgr2=0x7fe238155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.854+0000 7fe23db0d640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238144200 0x7fe238155220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c400 tx=0x7fe22800c8c0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.855+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 shutdown_connections 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.855+0000 7fe23db0d640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe22406b4d0 0x7fe22406d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.855+0000 7fe23db0d640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe23815eb30 0x7fe238157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.855+0000 7fe23db0d640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe238144200 0x7fe238155220 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.855+0000 7fe23db0d640 1 --2- 172.21.15.105:0/2350540588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe238140e70 0x7fe238156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.855+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 >> 172.21.15.105:0/2350540588 conn(0x7fe2380d0ae0 msgr2=0x7fe23815f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.855+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 shutdown_connections 2024-03-08T08:28:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.855+0000 7fe23db0d640 1 -- 172.21.15.105:0/2350540588 wait complete. 2024-03-08T08:28:29.869 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-08T08:28:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.973+0000 7f3e41c47640 1 Processor -- start 2024-03-08T08:28:29.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.973+0000 7f3e41c47640 1 -- start start 2024-03-08T08:28:29.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.974+0000 7f3e41c47640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3e3c06edf0 0x7f3e3c15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.974+0000 7f3e41c47640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c15aeb0 0x7f3e3c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.974+0000 7f3e41c47640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e3c15ebb0 con 0x7f3e3c15aad0 2024-03-08T08:28:29.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.974+0000 7f3e41c47640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3c15ee90 con 0x7f3e3c06edf0 2024-03-08T08:28:29.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.974+0000 7f3e41c47640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15f170 con 0x7f3e3c15aeb0 2024-03-08T08:28:29.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.974+0000 7f3e3b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3e3c15aad0 0x7f3e3c06e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39224/0 (socket says 172.21.15.105:39224) 2024-03-08T08:28:29.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.974+0000 7f3e3b7fe640 1 -- 172.21.15.105:0/1300088821 learned_addr learned my addr 172.21.15.105:0/1300088821 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:29.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e32ffd640 1 --2- 172.21.15.105:0/1300088821 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c06edf0 0x7f3e3c15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3bfff640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c15aeb0 0x7f3e3c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009312227 0 0) 0x7f3e3c15ebb0 con 0x7f3e3c15aad0 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e28003520 con 0x7f3e3c15aad0 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867234 0 0) 0x7f3e28003520 con 0x7f3e3c15aad0 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c15aeb0 msgr2=0x7f3e3c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c15aeb0 0x7f3e3c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c06edf0 msgr2=0x7f3e3c15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 --2- 172.21.15.105:0/1300088821 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c06edf0 0x7f3e3c15e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c15f450 con 0x7f3e3c15aad0 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f3e2c001690 con 0x7f3e3c15aad0 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 >> v1:172.21.15.105:6789/0 conn(0x7f3e3c15aad0 legacy=0x7f3e3c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e280039b0 0x7f3e28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c15aeb0 0x7f3e28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e28004d40 0x7f3e280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15ee90 con 0x7f3e280039b0 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15f170 con 0x7f3e28004d40 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2800b9c0 con 0x7f3e3c15aeb0 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e32ffd640 1 --2- 172.21.15.105:0/1300088821 >> v2:172.21.15.144:3300/0 conn(0x7f3e3c06edf0 0x7f3e3c15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e280039b0 0x7f3e28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3bfff640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e28004d40 0x7f3e280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e32ffd640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c15aeb0 0x7f3e28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3b7fe640 1 -- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c15aeb0 msgr2=0x7f3e28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c15aeb0 0x7f3e28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3b7fe640 1 -- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e28004d40 msgr2=0x7f3e280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e28004d40 0x7f3e280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3b7fe640 1 -- 172.21.15.105:0/1300088821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e28003520 con 0x7f3e280039b0 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.975+0000 7f3e3bfff640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e28004d40 0x7f3e280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:29.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.976+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e280039b0 0x7f3e28003dd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c210 tx=0x7f3e28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e07954f8fd539b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.976+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c007d70 con 0x7f3e280039b0 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.976+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c030070 con 0x7f3e280039b0 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.976+0000 7f3e3affd640 1 -- 172.21.15.105:0/1300088821 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c02cd60 con 0x7f3e280039b0 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.977+0000 7f3e41c47640 1 -- 172.21.15.105:0/1300088821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e280039b0 msgr2=0x7f3e28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.977+0000 7f3e41c47640 1 --2- 172.21.15.105:0/1300088821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e280039b0 0x7f3e28003dd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c210 tx=0x7f3e28005830 comp rx=0 tx=0).stop 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.977+0000 7f3e3bfff640 1 -- 172.21.15.105:0/1300088821 reap_dead start 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.977+0000 7f3e41c47640 1 -- 172.21.15.105:0/1300088821 shutdown_connections 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.977+0000 7f3e41c47640 1 -- 172.21.15.105:0/1300088821 >> 172.21.15.105:0/1300088821 conn(0x7f3e3c06bd90 msgr2=0x7f3e3c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.977+0000 7f3e41c47640 1 -- 172.21.15.105:0/1300088821 shutdown_connections 2024-03-08T08:28:29.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.977+0000 7f3e41c47640 1 -- 172.21.15.105:0/1300088821 wait complete. 2024-03-08T08:28:29.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.978+0000 7f3e41c47640 1 Processor -- start 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.978+0000 7f3e41c47640 1 -- start start 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e41c47640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e280039b0 0x7f3e3c140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e41c47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c151be0 0x7f3e3c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e41c47640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1528d0 0x7f3e3c155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e41c47640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c0728a0 con 0x7f3e3c151be0 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e41c47640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15b320 con 0x7f3e280039b0 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e41c47640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c15c760 con 0x7f3e3c1528d0 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e32ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c151be0 0x7f3e3c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e32ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c151be0 0x7f3e3c152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35000/0 (socket says 172.21.15.105:35000) 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e32ffd640 1 -- 172.21.15.105:0/1970987678 learned_addr learned my addr 172.21.15.105:0/1970987678 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e3bfff640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1528d0 0x7f3e3c155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e280039b0 0x7f3e3c140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e32ffd640 1 -- 172.21.15.105:0/1970987678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1528d0 msgr2=0x7f3e3c155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e32ffd640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1528d0 0x7f3e3c155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e32ffd640 1 -- 172.21.15.105:0/1970987678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e280039b0 msgr2=0x7f3e3c140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e32ffd640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e280039b0 0x7f3e3c140550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.979+0000 7f3e32ffd640 1 -- 172.21.15.105:0/1970987678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c150b80 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.980+0000 7f3e32ffd640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c151be0 0x7f3e3c152390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3e24011900 tx=0x7f3e24011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.980+0000 7f3e38ff9640 1 -- 172.21.15.105:0/1970987678 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400fbd0 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.980+0000 7f3e38ff9640 1 -- 172.21.15.105:0/1970987678 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2400fd70 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.980+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c150e20 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.980+0000 7f3e38ff9640 1 -- 172.21.15.105:0/1970987678 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24015660 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.980+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c1513e0 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.981+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c075120 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.981+0000 7f3e38ff9640 1 -- 172.21.15.105:0/1970987678 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e24015800 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.982+0000 7f3e38ff9640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e1c06b500 0x7f3e1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.982+0000 7f3e38ff9640 1 -- 172.21.15.105:0/1970987678 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f3e2408aa10 con 0x7f3e3c151be0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.982+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e1c06b500 0x7f3e1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:29.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.982+0000 7f3e3b7fe640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e1c06b500 0x7f3e1c06d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c034000 tx=0x7f3e2c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:29.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:29.986+0000 7f3e38ff9640 1 -- 172.21.15.105:0/1970987678 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e24019030 con 0x7f3e3c151be0 2024-03-08T08:28:30.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.159+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3e3c0673c0 con 0x7f3e3c151be0 2024-03-08T08:28:30.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.855+0000 7f3e38ff9640 1 -- 172.21.15.105:0/1970987678 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3e24059420 con 0x7f3e3c151be0 2024-03-08T08:28:30.857 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-03-08T08:28:30.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e1c06b500 msgr2=0x7f3e1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:30.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e1c06b500 0x7f3e1c06d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c034000 tx=0x7f3e2c007000 comp rx=0 tx=0).stop 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c151be0 msgr2=0x7f3e3c152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c151be0 0x7f3e3c152390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3e24011900 tx=0x7f3e24011dc0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 shutdown_connections 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e1c06b500 0x7f3e1c06d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e3c1528d0 0x7f3e3c155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e3c151be0 0x7f3e3c152390 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 --2- 172.21.15.105:0/1970987678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e280039b0 0x7f3e3c140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 >> 172.21.15.105:0/1970987678 conn(0x7f3e3c06bd90 msgr2=0x7f3e3c15cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 shutdown_connections 2024-03-08T08:28:30.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.858+0000 7f3e41c47640 1 -- 172.21.15.105:0/1970987678 wait complete. 2024-03-08T08:28:30.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-08T08:28:30.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.982+0000 7f76fba23640 1 Processor -- start 2024-03-08T08:28:30.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.983+0000 7f76fba23640 1 -- start start 2024-03-08T08:28:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.983+0000 7f76fba23640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f76f4072440 0x7f76f4071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.983+0000 7f76fba23640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f415af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:30.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.983+0000 7f76fba23640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f415b940 con 0x7f76f415b4a0 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.983+0000 7f76fba23640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f415bc20 con 0x7f76f4072440 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.983+0000 7f76fba23640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4070840 con 0x7f76f40b4d00 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f9fc0640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f76f415b4a0 0x7f76f406e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39230/0 (socket says 172.21.15.105:39230) 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f9fc0640 1 -- 172.21.15.105:0/2889880818 learned_addr learned my addr 172.21.15.105:0/2889880818 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f8fbe640 1 --2- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f97bf640 1 --2- 172.21.15.105:0/2889880818 >> v2:172.21.15.144:3300/0 conn(0x7f76f4072440 0x7f76f4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76ebfff640 1 -- 172.21.15.105:0/2889880818 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426643911 0 0) 0x7f76f415b940 con 0x7f76f415b4a0 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76ebfff640 1 -- 172.21.15.105:0/2889880818 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76e0003520 con 0x7f76f415b4a0 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f97bf640 1 -- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 msgr2=0x7f76f415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f97bf640 1 --2- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f97bf640 1 -- 172.21.15.105:0/2889880818 >> v1:172.21.15.105:6789/0 conn(0x7f76f415b4a0 legacy=0x7f76f406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76ebfff640 1 -- 172.21.15.105:0/2889880818 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1622965289 0 0) 0x7f76e0003520 con 0x7f76f415b4a0 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f97bf640 1 -- 172.21.15.105:0/2889880818 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4070b20 con 0x7f76f4072440 2024-03-08T08:28:30.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.984+0000 7f76f8fbe640 1 --2- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.985+0000 7f76f97bf640 1 --2- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4072440 0x7f76f4071020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009d90 tx=0x7f76ec02bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea85480ffea26e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.985+0000 7f76ebfff640 1 -- 172.21.15.105:0/2889880818 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec039070 con 0x7f76f4072440 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.985+0000 7f76ebfff640 1 -- 172.21.15.105:0/2889880818 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec02c8a0 con 0x7f76f4072440 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.985+0000 7f76ebfff640 1 -- 172.21.15.105:0/2889880818 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4001670 con 0x7f76f4072440 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.986+0000 7f76fba23640 1 -- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4072440 msgr2=0x7f76f4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.986+0000 7f76fba23640 1 --2- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4072440 0x7f76f4071020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009d90 tx=0x7f76ec02bd70 comp rx=0 tx=0).stop 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.986+0000 7f76fba23640 1 -- 172.21.15.105:0/2889880818 shutdown_connections 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.986+0000 7f76fba23640 1 --2- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.986+0000 7f76fba23640 1 --2- 172.21.15.105:0/2889880818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4072440 0x7f76f4071020 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.986+0000 7f76fba23640 1 -- 172.21.15.105:0/2889880818 >> 172.21.15.105:0/2889880818 conn(0x7f76f40b1740 msgr2=0x7f76f40b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.986+0000 7f76fba23640 1 -- 172.21.15.105:0/2889880818 shutdown_connections 2024-03-08T08:28:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.986+0000 7f76fba23640 1 -- 172.21.15.105:0/2889880818 wait complete. 2024-03-08T08:28:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.987+0000 7f76fba23640 1 Processor -- start 2024-03-08T08:28:30.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.987+0000 7f76fba23640 1 -- start start 2024-03-08T08:28:30.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.987+0000 7f76fba23640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76f4072440 0x7f76f4141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.987+0000 7f76fba23640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f40cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76fba23640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f415b4a0 0x7f76f406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76fba23640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40729a0 con 0x7f76f4072440 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76fba23640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4072820 con 0x7f76f415b4a0 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76fba23640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f406e7b0 con 0x7f76f40b4d00 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76f4072440 0x7f76f4141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76f4072440 0x7f76f4141750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35014/0 (socket says 172.21.15.105:35014) 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 -- 172.21.15.105:0/549876820 learned_addr learned my addr 172.21.15.105:0/549876820 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f8fbe640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f40cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f9fc0640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f415b4a0 0x7f76f406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 -- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 msgr2=0x7f76f40cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f40cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 -- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f415b4a0 msgr2=0x7f76f406cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:30.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f415b4a0 0x7f76f406cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 -- 172.21.15.105:0/549876820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f40ca340 con 0x7f76f4072440 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f8fbe640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f9fc0640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f415b4a0 0x7f76f406cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76f97bf640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76f4072440 0x7f76f4141750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009d90 tx=0x7f76ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76e9ffb640 1 -- 172.21.15.105:0/549876820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec039050 con 0x7f76f4072440 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76e9ffb640 1 -- 172.21.15.105:0/549876820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec045070 con 0x7f76f4072440 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f40ca620 con 0x7f76f4072440 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.988+0000 7f76e9ffb640 1 -- 172.21.15.105:0/549876820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec009450 con 0x7f76f4072440 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.989+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f40cb640 con 0x7f76f4072440 2024-03-08T08:28:30.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.989+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f4075120 con 0x7f76f4072440 2024-03-08T08:28:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.990+0000 7f76e9ffb640 1 -- 172.21.15.105:0/549876820 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76ec040690 con 0x7f76f4072440 2024-03-08T08:28:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.990+0000 7f76e9ffb640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f76d806b4d0 0x7f76d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.991+0000 7f76e9ffb640 1 -- 172.21.15.105:0/549876820 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f76ec0aca00 con 0x7f76f4072440 2024-03-08T08:28:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.991+0000 7f76f8fbe640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f76d806b4d0 0x7f76d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:30.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.991+0000 7f76f8fbe640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f76d806b4d0 0x7f76d806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f76f40ca160 tx=0x7f76f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:30.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:30.995+0000 7f76e9ffb640 1 -- 172.21.15.105:0/549876820 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76ec07b8e0 con 0x7f76f4072440 2024-03-08T08:28:31.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.200+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f76f40b4180 con 0x7f76f4072440 2024-03-08T08:28:31.877 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 4 and data pool 5 2024-03-08T08:28:31.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.875+0000 7f76e9ffb640 1 -- 172.21.15.105:0/549876820 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f76ec07b1e0 con 0x7f76f4072440 2024-03-08T08:28:31.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.879+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f76d806b4d0 msgr2=0x7f76d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:31.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.879+0000 7f76fba23640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f76d806b4d0 0x7f76d806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f76f40ca160 tx=0x7f76f0008040 comp rx=0 tx=0).stop 2024-03-08T08:28:31.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.880+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76f4072440 msgr2=0x7f76f4141750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:31.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.880+0000 7f76fba23640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76f4072440 0x7f76f4141750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009d90 tx=0x7f76ec004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:31.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.880+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 shutdown_connections 2024-03-08T08:28:31.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.881+0000 7f76fba23640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f76d806b4d0 0x7f76d806d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:31.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.881+0000 7f76fba23640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f415b4a0 0x7f76f406cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:31.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.881+0000 7f76fba23640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76f40b4d00 0x7f76f40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:31.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.881+0000 7f76fba23640 1 --2- 172.21.15.105:0/549876820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76f4072440 0x7f76f4141750 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:31.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.881+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 >> 172.21.15.105:0/549876820 conn(0x7f76f40b1740 msgr2=0x7f76f4151540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:31.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.884+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 shutdown_connections 2024-03-08T08:28:31.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:31.884+0000 7f76fba23640 1 -- 172.21.15.105:0/549876820 wait complete. 2024-03-08T08:28:31.906 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:32.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.004+0000 7fd22482e640 1 Processor -- start 2024-03-08T08:28:32.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.004+0000 7fd22482e640 1 -- start start 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd22482e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd220161470 0x7fd22006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd22482e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd22006bcb0 0x7fd2200d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd22482e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2200d48a0 con 0x7fd220161850 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd22482e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2200d2ba0 con 0x7fd22006bcb0 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd22482e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2200d4b80 con 0x7fd220161470 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd21ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd22006bcb0 0x7fd2200d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd21ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd22006bcb0 0x7fd2200d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35022/0 (socket says 172.21.15.105:35022) 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd21ed9d640 1 -- 172.21.15.105:0/4084797859 learned_addr learned my addr 172.21.15.105:0/4084797859 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.005+0000 7fd21e59c640 1 --2- 172.21.15.105:0/4084797859 >> v2:172.21.15.144:3300/0 conn(0x7fd220161470 0x7fd22006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.006+0000 7fd21ed9d640 1 -- 172.21.15.105:0/4084797859 >> v2:172.21.15.144:3300/0 conn(0x7fd220161470 msgr2=0x7fd22006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.006+0000 7fd21ed9d640 1 --2- 172.21.15.105:0/4084797859 >> v2:172.21.15.144:3300/0 conn(0x7fd220161470 0x7fd22006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.006+0000 7fd21ed9d640 1 -- 172.21.15.105:0/4084797859 >> v1:172.21.15.144:6789/0 conn(0x7fd220161850 legacy=0x7fd22006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.006+0000 7fd21ed9d640 1 -- 172.21.15.105:0/4084797859 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22006c170 con 0x7fd22006bcb0 2024-03-08T08:28:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.006+0000 7fd21ed9d640 1 --2- 172.21.15.105:0/4084797859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd22006bcb0 0x7fd2200d2660 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd22008c850 tx=0x7fd21802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46ecf8f1c4196ba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.006+0000 7fd21dd9b640 1 -- 172.21.15.105:0/4084797859 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21802ca80 con 0x7fd22006bcb0 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.006+0000 7fd21dd9b640 1 -- 172.21.15.105:0/4084797859 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21802cc20 con 0x7fd22006bcb0 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.006+0000 7fd21dd9b640 1 -- 172.21.15.105:0/4084797859 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180343f0 con 0x7fd22006bcb0 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.007+0000 7fd22482e640 1 -- 172.21.15.105:0/4084797859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd22006bcb0 msgr2=0x7fd2200d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.007+0000 7fd22482e640 1 --2- 172.21.15.105:0/4084797859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd22006bcb0 0x7fd2200d2660 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd22008c850 tx=0x7fd21802bee0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.007+0000 7fd22482e640 1 -- 172.21.15.105:0/4084797859 shutdown_connections 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.007+0000 7fd22482e640 1 --2- 172.21.15.105:0/4084797859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd22006bcb0 0x7fd2200d2660 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.007+0000 7fd22482e640 1 --2- 172.21.15.105:0/4084797859 >> v2:172.21.15.144:3300/0 conn(0x7fd220161470 0x7fd22006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.007+0000 7fd22482e640 1 -- 172.21.15.105:0/4084797859 >> 172.21.15.105:0/4084797859 conn(0x7fd2200d3ce0 msgr2=0x7fd2200d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.007+0000 7fd22482e640 1 -- 172.21.15.105:0/4084797859 shutdown_connections 2024-03-08T08:28:32.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.007+0000 7fd22482e640 1 -- 172.21.15.105:0/4084797859 wait complete. 2024-03-08T08:28:32.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.008+0000 7fd22482e640 1 Processor -- start 2024-03-08T08:28:32.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.008+0000 7fd22482e640 1 -- start start 2024-03-08T08:28:32.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd22482e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22006bcb0 0x7fd220155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd22482e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd220161470 0x7fd2201f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd22482e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd220161850 0x7fd22015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd22482e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22006cf40 con 0x7fd220161470 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd22482e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22006d370 con 0x7fd22006bcb0 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd22482e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201f4d50 con 0x7fd220161850 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd215d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd220161470 0x7fd2201f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd215d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd220161470 0x7fd2201f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35030/0 (socket says 172.21.15.105:35030) 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd215d9b640 1 -- 172.21.15.105:0/609913365 learned_addr learned my addr 172.21.15.105:0/609913365 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd21e59c640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22006bcb0 0x7fd220155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.009+0000 7fd21ed9d640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd220161850 0x7fd22015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd215d9b640 1 -- 172.21.15.105:0/609913365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd220161850 msgr2=0x7fd22015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd215d9b640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd220161850 0x7fd22015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd215d9b640 1 -- 172.21.15.105:0/609913365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22006bcb0 msgr2=0x7fd220155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd215d9b640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22006bcb0 0x7fd220155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd215d9b640 1 -- 172.21.15.105:0/609913365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2200b9800 con 0x7fd220161470 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd215d9b640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd220161470 0x7fd2201f4810 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd20800c440 tx=0x7fd20800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd2177fe640 1 -- 172.21.15.105:0/609913365 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080052d0 con 0x7fd220161470 2024-03-08T08:28:32.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd2177fe640 1 -- 172.21.15.105:0/609913365 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208005470 con 0x7fd220161470 2024-03-08T08:28:32.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd2177fe640 1 -- 172.21.15.105:0/609913365 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208007260 con 0x7fd220161470 2024-03-08T08:28:32.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2200b9520 con 0x7fd220161470 2024-03-08T08:28:32.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.010+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2200b6c80 con 0x7fd220161470 2024-03-08T08:28:32.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.011+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd220075120 con 0x7fd220161470 2024-03-08T08:28:32.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.011+0000 7fd2177fe640 1 -- 172.21.15.105:0/609913365 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2080074c0 con 0x7fd220161470 2024-03-08T08:28:32.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.012+0000 7fd2177fe640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20006b4d0 0x7fd20006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.012+0000 7fd2177fe640 1 -- 172.21.15.105:0/609913365 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2080895d0 con 0x7fd220161470 2024-03-08T08:28:32.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.012+0000 7fd21e59c640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20006b4d0 0x7fd20006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.012+0000 7fd21e59c640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20006b4d0 0x7fd20006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd210006fd0 tx=0x7fd210008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.016+0000 7fd2177fe640 1 -- 172.21.15.105:0/609913365 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd208057e00 con 0x7fd220161470 2024-03-08T08:28:32.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.187+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd22006c940 con 0x7fd220161470 2024-03-08T08:28:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.188+0000 7fd2177fe640 1 -- 172.21.15.105:0/609913365 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16694 (secure 0 0 0) 0x7fd20805bea0 con 0x7fd220161470 2024-03-08T08:28:32.191 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:32.191 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":32,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:31.202791+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:28:29.680862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T08:28:30.179255+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:32.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20006b4d0 msgr2=0x7fd20006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20006b4d0 0x7fd20006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd210006fd0 tx=0x7fd210008040 comp rx=0 tx=0).stop 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd220161470 msgr2=0x7fd2201f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd220161470 0x7fd2201f4810 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd20800c440 tx=0x7fd20800c900 comp rx=0 tx=0).stop 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 shutdown_connections 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20006b4d0 0x7fd20006d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd220161850 0x7fd22015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd220161470 0x7fd2201f4810 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 --2- 172.21.15.105:0/609913365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22006bcb0 0x7fd220155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 >> 172.21.15.105:0/609913365 conn(0x7fd2200d3ce0 msgr2=0x7fd22006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 shutdown_connections 2024-03-08T08:28:32.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.190+0000 7fd22482e640 1 -- 172.21.15.105:0/609913365 wait complete. 2024-03-08T08:28:32.203 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:32.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.301+0000 7f7a102cd640 1 Processor -- start 2024-03-08T08:28:32.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.301+0000 7f7a102cd640 1 -- start start 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a102cd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a080cf290 0x7f7a080b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a102cd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a080cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a102cd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a080b4920 con 0x7f7a080b4000 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a102cd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a080cd9e0 con 0x7f7a080cf290 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a102cd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08072120 con 0x7f7a0806cf90 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a0e069640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7a080b4000 0x7f7a0815b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:52924/0 (socket says 172.21.15.105:52924) 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a0d868640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a080cf290 0x7f7a080b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a0e86a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a080cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.302+0000 7f7a0e069640 1 -- 172.21.15.105:0/566803182 learned_addr learned my addr 172.21.15.105:0/566803182 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.303+0000 7f7a0d067640 1 -- 172.21.15.105:0/566803182 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607178257 0 0) 0x7f7a080b4920 con 0x7f7a080b4000 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.303+0000 7f7a0d067640 1 -- 172.21.15.105:0/566803182 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f4003520 con 0x7f7a080b4000 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.303+0000 7f7a0e86a640 1 -- 172.21.15.105:0/566803182 >> v2:172.21.15.144:3300/0 conn(0x7f7a080cf290 msgr2=0x7f7a080b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:32.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.303+0000 7f7a0e86a640 1 --2- 172.21.15.105:0/566803182 >> v2:172.21.15.144:3300/0 conn(0x7f7a080cf290 0x7f7a080b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.303+0000 7f7a0e86a640 1 -- 172.21.15.105:0/566803182 >> v1:172.21.15.144:6789/0 conn(0x7f7a080b4000 legacy=0x7f7a0815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.303+0000 7f7a0e86a640 1 -- 172.21.15.105:0/566803182 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a08072400 con 0x7f7a0806cf90 2024-03-08T08:28:32.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.303+0000 7f7a0e86a640 1 --2- 172.21.15.105:0/566803182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a080cde70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7a000097a0 tx=0x7f7a0002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0edcf6615e51cd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a0d067640 1 -- 172.21.15.105:0/566803182 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00035020 con 0x7f7a0806cf90 2024-03-08T08:28:32.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a0d067640 1 -- 172.21.15.105:0/566803182 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a00031830 con 0x7f7a0806cf90 2024-03-08T08:28:32.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a0d067640 1 -- 172.21.15.105:0/566803182 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00031c00 con 0x7f7a0806cf90 2024-03-08T08:28:32.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a102cd640 1 -- 172.21.15.105:0/566803182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 msgr2=0x7f7a080cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a102cd640 1 --2- 172.21.15.105:0/566803182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a080cde70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7a000097a0 tx=0x7f7a0002b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a102cd640 1 -- 172.21.15.105:0/566803182 shutdown_connections 2024-03-08T08:28:32.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a102cd640 1 --2- 172.21.15.105:0/566803182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a080cde70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a102cd640 1 --2- 172.21.15.105:0/566803182 >> v2:172.21.15.144:3300/0 conn(0x7f7a080cf290 0x7f7a080b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a102cd640 1 -- 172.21.15.105:0/566803182 >> 172.21.15.105:0/566803182 conn(0x7f7a080b1740 msgr2=0x7f7a080b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:32.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a102cd640 1 -- 172.21.15.105:0/566803182 shutdown_connections 2024-03-08T08:28:32.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.304+0000 7f7a102cd640 1 -- 172.21.15.105:0/566803182 wait complete. 2024-03-08T08:28:32.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.305+0000 7f7a102cd640 1 Processor -- start 2024-03-08T08:28:32.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.305+0000 7f7a102cd640 1 -- start start 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a102cd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a0813c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a102cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a080b4000 0x7f7a081523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a102cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a080cf290 0x7f7a08152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a102cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0815a500 con 0x7f7a080cf290 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a102cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080726e0 con 0x7f7a080b4000 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a102cd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0806d420 con 0x7f7a0806cf90 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a0e86a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a080cf290 0x7f7a08152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a0e86a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a080cf290 0x7f7a08152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35046/0 (socket says 172.21.15.105:35046) 2024-03-08T08:28:32.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a0e86a640 1 -- 172.21.15.105:0/3103262938 learned_addr learned my addr 172.21.15.105:0/3103262938 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a0e069640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a0813c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.306+0000 7f7a0d868640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a080b4000 0x7f7a081523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a0e86a640 1 -- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 msgr2=0x7f7a0813c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a0e86a640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a0813c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a0e86a640 1 -- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a080b4000 msgr2=0x7f7a081523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a0e86a640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a080b4000 0x7f7a081523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a0e86a640 1 -- 172.21.15.105:0/3103262938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0814ca70 con 0x7f7a080cf290 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a0d868640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a080b4000 0x7f7a081523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a0e86a640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a080cf290 0x7f7a08152930 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7a000097a0 tx=0x7f7a000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f79feffd640 1 -- 172.21.15.105:0/3103262938 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00035040 con 0x7f7a080cf290 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f79feffd640 1 -- 172.21.15.105:0/3103262938 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0003b840 con 0x7f7a080cf290 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f79feffd640 1 -- 172.21.15.105:0/3103262938 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0003bc30 con 0x7f7a080cf290 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0813cb60 con 0x7f7a080cf290 2024-03-08T08:28:32.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.307+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a08155930 con 0x7f7a080cf290 2024-03-08T08:28:32.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.308+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a08075120 con 0x7f7a080cf290 2024-03-08T08:28:32.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.308+0000 7f79feffd640 1 -- 172.21.15.105:0/3103262938 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a00038020 con 0x7f7a080cf290 2024-03-08T08:28:32.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.309+0000 7f79feffd640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f79ec06b4d0 0x7f79ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.309+0000 7f7a0e069640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f79ec06b4d0 0x7f79ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.309+0000 7f79feffd640 1 -- 172.21.15.105:0/3103262938 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a000ac2c0 con 0x7f7a080cf290 2024-03-08T08:28:32.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.309+0000 7f7a0e069640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f79ec06b4d0 0x7f79ec06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a04005c00 tx=0x7f7a04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.313+0000 7f79feffd640 1 -- 172.21.15.105:0/3103262938 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a0813cb60 con 0x7f7a080cf290 2024-03-08T08:28:32.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.485+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a08074800 con 0x7f7a080cf290 2024-03-08T08:28:32.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.486+0000 7f79feffd640 1 -- 172.21.15.105:0/3103262938 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16694 (secure 0 0 0) 0x7f7a0007b140 con 0x7f7a080cf290 2024-03-08T08:28:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:32.489 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":32,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:31.202791+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:28:29.680862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T08:28:30.179255+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f79ec06b4d0 msgr2=0x7f79ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f79ec06b4d0 0x7f79ec06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a04005c00 tx=0x7f7a04009040 comp rx=0 tx=0).stop 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a080cf290 msgr2=0x7f7a08152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a080cf290 0x7f7a08152930 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7a000097a0 tx=0x7f7a000046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 shutdown_connections 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f79ec06b4d0 0x7f79ec06d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a080cf290 0x7f7a08152930 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a080b4000 0x7f7a081523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 --2- 172.21.15.105:0/3103262938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0806cf90 0x7f7a0813c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.488+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 >> 172.21.15.105:0/3103262938 conn(0x7f7a080b1740 msgr2=0x7f7a08158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.489+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 shutdown_connections 2024-03-08T08:28:32.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.489+0000 7f7a102cd640 1 -- 172.21.15.105:0/3103262938 wait complete. 2024-03-08T08:28:32.501 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-08T08:28:32.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.602+0000 7feaecc83640 1 Processor -- start 2024-03-08T08:28:32.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.602+0000 7feaecc83640 1 -- start start 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feaecc83640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d43e0 0x7feae80b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feaecc83640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feae80d2910 0x7feae815de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feaecc83640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae80d48f0 con 0x7feae80d2530 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feaecc83640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae80d4bb0 con 0x7feae80d2910 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feaecc83640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae80ceb30 con 0x7feae80d43e0 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feae80d2530 0x7feae80b3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39232/0 (socket says 172.21.15.105:39232) 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae659c640 1 -- 172.21.15.105:0/1434745434 learned_addr learned my addr 172.21.15.105:0/1434745434 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae6d9d640 1 --2- 172.21.15.105:0/1434745434 >> v2:172.21.15.105:3300/0 conn(0x7feae80d2910 0x7feae815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae5d9b640 1 --2- 172.21.15.105:0/1434745434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d43e0 0x7feae80b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae559a640 1 -- 172.21.15.105:0/1434745434 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696172764 0 0) 0x7feae80d48f0 con 0x7feae80d2530 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae559a640 1 -- 172.21.15.105:0/1434745434 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead0003520 con 0x7feae80d2530 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae6d9d640 1 -- 172.21.15.105:0/1434745434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d43e0 msgr2=0x7feae80b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:32.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae6d9d640 1 --2- 172.21.15.105:0/1434745434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d43e0 0x7feae80b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae6d9d640 1 -- 172.21.15.105:0/1434745434 >> v1:172.21.15.105:6789/0 conn(0x7feae80d2530 legacy=0x7feae80b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae6d9d640 1 -- 172.21.15.105:0/1434745434 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80cee10 con 0x7feae80d2910 2024-03-08T08:28:32.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.603+0000 7feae5d9b640 1 --2- 172.21.15.105:0/1434745434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d43e0 0x7feae80b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:32.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.604+0000 7feae6d9d640 1 --2- 172.21.15.105:0/1434745434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d2910 0x7feae815de00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feae0009d90 tx=0x7feae002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c035e941a5f54715 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.604+0000 7feae559a640 1 -- 172.21.15.105:0/1434745434 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0039070 con 0x7feae80d2910 2024-03-08T08:28:32.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.604+0000 7feae559a640 1 -- 172.21.15.105:0/1434745434 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae002cb30 con 0x7feae80d2910 2024-03-08T08:28:32.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.604+0000 7feae559a640 1 -- 172.21.15.105:0/1434745434 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0034440 con 0x7feae80d2910 2024-03-08T08:28:32.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.605+0000 7feaecc83640 1 -- 172.21.15.105:0/1434745434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d2910 msgr2=0x7feae815de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.605+0000 7feaecc83640 1 --2- 172.21.15.105:0/1434745434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d2910 0x7feae815de00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feae0009d90 tx=0x7feae002bd80 comp rx=0 tx=0).stop 2024-03-08T08:28:32.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.605+0000 7feaecc83640 1 -- 172.21.15.105:0/1434745434 shutdown_connections 2024-03-08T08:28:32.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.605+0000 7feaecc83640 1 --2- 172.21.15.105:0/1434745434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d2910 0x7feae815de00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.605+0000 7feaecc83640 1 --2- 172.21.15.105:0/1434745434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d43e0 0x7feae80b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.605+0000 7feaecc83640 1 -- 172.21.15.105:0/1434745434 >> 172.21.15.105:0/1434745434 conn(0x7feae809cfb0 msgr2=0x7feae80d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:32.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.605+0000 7feaecc83640 1 -- 172.21.15.105:0/1434745434 shutdown_connections 2024-03-08T08:28:32.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.605+0000 7feaecc83640 1 -- 172.21.15.105:0/1434745434 wait complete. 2024-03-08T08:28:32.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.606+0000 7feaecc83640 1 Processor -- start 2024-03-08T08:28:32.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.606+0000 7feaecc83640 1 -- start start 2024-03-08T08:28:32.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feaecc83640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d2530 0x7feae81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feaecc83640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feae80d2910 0x7feae81f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feaecc83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d43e0 0x7feae815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feaecc83640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae815e340 con 0x7feae80d43e0 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feaecc83640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae80b3080 con 0x7feae80d2910 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feaecc83640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae815e4c0 con 0x7feae80d2530 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d43e0 0x7feae815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d43e0 0x7feae815c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35060/0 (socket says 172.21.15.105:35060) 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 -- 172.21.15.105:0/793215096 learned_addr learned my addr 172.21.15.105:0/793215096 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae659c640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d2530 0x7feae81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae5d9b640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feae80d2910 0x7feae81f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 -- 172.21.15.105:0/793215096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d2530 msgr2=0x7feae81520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d2530 0x7feae81520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 -- 172.21.15.105:0/793215096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feae80d2910 msgr2=0x7feae81f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feae80d2910 0x7feae81f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 -- 172.21.15.105:0/793215096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae81f7be0 con 0x7feae80d43e0 2024-03-08T08:28:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.607+0000 7feae6d9d640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d43e0 0x7feae815c380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feae0009d90 tx=0x7feae0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.608+0000 7fead6ffd640 1 -- 172.21.15.105:0/793215096 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0039050 con 0x7feae80d43e0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.608+0000 7fead6ffd640 1 -- 172.21.15.105:0/793215096 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0045070 con 0x7feae80d43e0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.608+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae81f7ec0 con 0x7feae80d43e0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.608+0000 7fead6ffd640 1 -- 172.21.15.105:0/793215096 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae003e850 con 0x7feae80d43e0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.608+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae80b7c00 con 0x7feae80d43e0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.609+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae8075120 con 0x7feae80d43e0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.609+0000 7fead6ffd640 1 -- 172.21.15.105:0/793215096 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feae003e9f0 con 0x7feae80d43e0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.609+0000 7fead6ffd640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feac806b4d0 0x7feac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.609+0000 7fead6ffd640 1 -- 172.21.15.105:0/793215096 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feae00acf30 con 0x7feae80d43e0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.609+0000 7feae659c640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feac806b4d0 0x7feac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.610+0000 7feae659c640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feac806b4d0 0x7feac806d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fead8000f60 tx=0x7fead8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.614+0000 7fead6ffd640 1 -- 172.21.15.105:0/793215096 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feae007bed0 con 0x7feae80d43e0 2024-03-08T08:28:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.820+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feae808dd60 con 0x7feae80d43e0 2024-03-08T08:28:32.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.876+0000 7fead6ffd640 1 -- 172.21.15.105:0/793215096 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7feae007b7d0 con 0x7feae80d43e0 2024-03-08T08:28:32.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feac806b4d0 msgr2=0x7feac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feac806b4d0 0x7feac806d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fead8000f60 tx=0x7fead8009040 comp rx=0 tx=0).stop 2024-03-08T08:28:32.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d43e0 msgr2=0x7feae815c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d43e0 0x7feae815c380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feae0009d90 tx=0x7feae0004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 shutdown_connections 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7feac806b4d0 0x7feac806d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feae80d43e0 0x7feae815c380 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feae80d2910 0x7feae81f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 --2- 172.21.15.105:0/793215096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feae80d2530 0x7feae81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 >> 172.21.15.105:0/793215096 conn(0x7feae809cfb0 msgr2=0x7feae806c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 shutdown_connections 2024-03-08T08:28:32.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.878+0000 7feaecc83640 1 -- 172.21.15.105:0/793215096 wait complete. 2024-03-08T08:28:32.892 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-08T08:28:32.892 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-08T08:28:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.993+0000 7fcb4f7d7640 1 Processor -- start 2024-03-08T08:28:32.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.993+0000 7fcb4f7d7640 1 -- start start 2024-03-08T08:28:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb4f7d7640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcb481469e0 0x7fcb4806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:32.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb4f7d7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb480adcd0 con 0x7fcb4806ac80 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb4f7d7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4806deb0 con 0x7fcb480ad870 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb4f7d7640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4806e190 con 0x7fcb481469e0 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb4dd74640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcb4806ac80 0x7fcb480b4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39244/0 (socket says 172.21.15.105:39244) 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb4dd74640 1 -- 172.21.15.105:0/2617384510 learned_addr learned my addr 172.21.15.105:0/2617384510 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587175880 0 0) 0x7fcb480adcd0 con 0x7fcb4806ac80 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb34003520 con 0x7fcb4806ac80 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb4d573640 1 --2- 172.21.15.105:0/2617384510 >> v2:172.21.15.144:3300/0 conn(0x7fcb481469e0 0x7fcb4806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695807213 0 0) 0x7fcb34003520 con 0x7fcb4806ac80 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 >> v2:172.21.15.144:3300/0 conn(0x7fcb481469e0 msgr2=0x7fcb4806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 --2- 172.21.15.105:0/2617384510 >> v2:172.21.15.144:3300/0 conn(0x7fcb481469e0 0x7fcb4806fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 >> v1:172.21.15.144:6789/0 conn(0x7fcb480ad870 legacy=0x7fcb4806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480b42f0 con 0x7fcb4806ac80 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fcb40001a10 con 0x7fcb4806ac80 2024-03-08T08:28:32.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb4d573640 1 --2- 172.21.15.105:0/2617384510 >> v2:172.21.15.144:3300/0 conn(0x7fcb481469e0 0x7fcb4806fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 >> v1:172.21.15.105:6789/0 conn(0x7fcb4806ac80 legacy=0x7fcb480b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb340039b0 0x7fcb34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb34004800 0x7fcb34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb340052e0 0x7fcb3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4806e190 con 0x7fcb340039b0 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3400c1b0 con 0x7fcb34004800 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.994+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3400c3b0 con 0x7fcb340052e0 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4d573640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb340039b0 0x7fcb34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4cd72640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb34004800 0x7fcb34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4dd74640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb340052e0 0x7fcb3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4d573640 1 -- 172.21.15.105:0/2617384510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb340052e0 msgr2=0x7fcb3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4d573640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb340052e0 0x7fcb3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4d573640 1 -- 172.21.15.105:0/2617384510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb34004800 msgr2=0x7fcb34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4d573640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb34004800 0x7fcb34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4d573640 1 -- 172.21.15.105:0/2617384510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34003520 con 0x7fcb340039b0 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.995+0000 7fcb4dd74640 1 -- 172.21.15.105:0/2617384510 reap_dead start 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb4d573640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb340039b0 0x7fcb34003dd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcb44006fd0 tx=0x7fcb34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9191184ed363d4b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400f040 con 0x7fcb340039b0 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb44004030 con 0x7fcb340039b0 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb3ffff640 1 -- 172.21.15.105:0/2617384510 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb44004440 con 0x7fcb340039b0 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/2617384510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb340039b0 msgr2=0x7fcb34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb4f7d7640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb340039b0 0x7fcb34003dd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcb44006fd0 tx=0x7fcb34005830 comp rx=0 tx=0).stop 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/2617384510 shutdown_connections 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb4f7d7640 1 --2- 172.21.15.105:0/2617384510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb340039b0 0x7fcb34003dd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/2617384510 >> 172.21.15.105:0/2617384510 conn(0x7fcb480b2670 msgr2=0x7fcb480b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/2617384510 shutdown_connections 2024-03-08T08:28:33.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.996+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/2617384510 wait complete. 2024-03-08T08:28:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.997+0000 7fcb4f7d7640 1 Processor -- start 2024-03-08T08:28:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.997+0000 7fcb4f7d7640 1 -- start start 2024-03-08T08:28:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4f7d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb340039b0 0x7fcb481f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4f7d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb4815d220 0x7fcb480cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4f7d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb481f6aa0 0x7fcb48141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:33.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4f7d7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb48146dc0 con 0x7fcb481f6aa0 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4f7d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4806e470 con 0x7fcb340039b0 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4f7d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4806fd60 con 0x7fcb4815d220 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4dd74640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb481f6aa0 0x7fcb48141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4dd74640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb481f6aa0 0x7fcb48141da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35084/0 (socket says 172.21.15.105:35084) 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4dd74640 1 -- 172.21.15.105:0/1021404477 learned_addr learned my addr 172.21.15.105:0/1021404477 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4d573640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb340039b0 0x7fcb481f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4cd72640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb4815d220 0x7fcb480cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4dd74640 1 -- 172.21.15.105:0/1021404477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb4815d220 msgr2=0x7fcb480cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4dd74640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb4815d220 0x7fcb480cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4dd74640 1 -- 172.21.15.105:0/1021404477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb340039b0 msgr2=0x7fcb481f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4dd74640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb340039b0 0x7fcb481f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.998+0000 7fcb4dd74640 1 -- 172.21.15.105:0/1021404477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4815e640 con 0x7fcb481f6aa0 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.999+0000 7fcb4dd74640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb481f6aa0 0x7fcb48141da0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb480adea0 tx=0x7fcb40007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.999+0000 7fcb3dffb640 1 -- 172.21.15.105:0/1021404477 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000ab60 con 0x7fcb481f6aa0 2024-03-08T08:28:33.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.999+0000 7fcb3dffb640 1 -- 172.21.15.105:0/1021404477 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb40018070 con 0x7fcb481f6aa0 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.999+0000 7fcb3dffb640 1 -- 172.21.15.105:0/1021404477 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400133f0 con 0x7fcb481f6aa0 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.999+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4815e8e0 con 0x7fcb481f6aa0 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:32.999+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4815eea0 con 0x7fcb481f6aa0 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.000+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4815f0a0 con 0x7fcb481f6aa0 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.000+0000 7fcb3dffb640 1 -- 172.21.15.105:0/1021404477 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb40013590 con 0x7fcb481f6aa0 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.001+0000 7fcb3dffb640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.001+0000 7fcb3dffb640 1 -- 172.21.15.105:0/1021404477 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb4008a850 con 0x7fcb481f6aa0 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.001+0000 7fcb4d573640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:33.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.001+0000 7fcb4d573640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcb44035000 tx=0x7fcb4403a040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:33.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.005+0000 7fcb3dffb640 1 -- 172.21.15.105:0/1021404477 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb4815f0a0 con 0x7fcb481f6aa0 2024-03-08T08:28:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.211+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcb48075120 con 0x7fcb481f6aa0 2024-03-08T08:28:33.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.884+0000 7fcb3dffb640 1 -- 172.21.15.105:0/1021404477 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcb40059640 con 0x7fcb481f6aa0 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.886+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb2c06b4d0 msgr2=0x7fcb2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.886+0000 7fcb4f7d7640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcb44035000 tx=0x7fcb4403a040 comp rx=0 tx=0).stop 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.886+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb481f6aa0 msgr2=0x7fcb48141da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.886+0000 7fcb4f7d7640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb481f6aa0 0x7fcb48141da0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb480adea0 tx=0x7fcb40007b00 comp rx=0 tx=0).stop 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.887+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 shutdown_connections 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.887+0000 7fcb4f7d7640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcb2c06b4d0 0x7fcb2c06d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.887+0000 7fcb4f7d7640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb481f6aa0 0x7fcb48141da0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.887+0000 7fcb4f7d7640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb4815d220 0x7fcb480cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.887+0000 7fcb4f7d7640 1 --2- 172.21.15.105:0/1021404477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb340039b0 0x7fcb481f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.887+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 >> 172.21.15.105:0/1021404477 conn(0x7fcb480b2670 msgr2=0x7fcb480b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.887+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 shutdown_connections 2024-03-08T08:28:33.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:33.887+0000 7fcb4f7d7640 1 -- 172.21.15.105:0/1021404477 wait complete. 2024-03-08T08:28:33.901 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-08T08:28:33.901 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:28:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.002+0000 7fc508457640 1 Processor -- start 2024-03-08T08:28:34.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.002+0000 7fc508457640 1 -- start start 2024-03-08T08:28:34.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc508457640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc500072440 0x7fc500071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc508457640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50015b670 con 0x7fc50006e0a0 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc508457640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50015b940 con 0x7fc5000b4d00 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc508457640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50015bc20 con 0x7fc500072440 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc5069f4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc50006e0a0 0x7fc5000cdd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38048/0 (socket says 172.21.15.105:38048) 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc5069f4640 1 -- 172.21.15.105:0/1280668109 learned_addr learned my addr 172.21.15.105:0/1280668109 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1280668109 >> v2:172.21.15.144:3300/0 conn(0x7fc500072440 0x7fc500071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263422232 0 0) 0x7fc50015b670 con 0x7fc50006e0a0 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.003+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4ec003520 con 0x7fc50006e0a0 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411383511 0 0) 0x7fc50015b940 con 0x7fc5000b4d00 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc50015b670 con 0x7fc5000b4d00 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1094118884 0 0) 0x7fc4ec003520 con 0x7fc50006e0a0 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 >> v2:172.21.15.144:3300/0 conn(0x7fc500072440 msgr2=0x7fc500071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 --2- 172.21.15.105:0/1280668109 >> v2:172.21.15.144:3300/0 conn(0x7fc500072440 0x7fc500071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 >> v1:172.21.15.144:6789/0 conn(0x7fc5000b4d00 legacy=0x7fc50015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500070840 con 0x7fc50006e0a0 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fc4f80019f0 con 0x7fc50006e0a0 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 >> v1:172.21.15.105:6789/0 conn(0x7fc50006e0a0 legacy=0x7fc5000cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4ec003b40 0x7fc4ec003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4ec004800 0x7fc4ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec0051c0 0x7fc4ec00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50015bc20 con 0x7fc4ec003b40 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec00e250 con 0x7fc4ec004800 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec00e450 con 0x7fc4ec0051c0 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4ec003b40 0x7fc4ec003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5059f2640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4ec004800 0x7fc4ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5069f4640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec0051c0 0x7fc4ec00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5061f3640 1 -- 172.21.15.105:0/1280668109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec0051c0 msgr2=0x7fc4ec00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec0051c0 0x7fc4ec00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5061f3640 1 -- 172.21.15.105:0/1280668109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4ec004800 msgr2=0x7fc4ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4ec004800 0x7fc4ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5061f3640 1 -- 172.21.15.105:0/1280668109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec003520 con 0x7fc4ec003b40 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.004+0000 7fc5069f4640 1 -- 172.21.15.105:0/1280668109 reap_dead start 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.005+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4ec003b40 0x7fc4ec003f40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009ac0 tx=0x7fc4ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b317fbdb0ad8478 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.005+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc039070 con 0x7fc4ec003b40 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.005+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc02c6b0 con 0x7fc4ec003b40 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.005+0000 7fc5051f1640 1 -- 172.21.15.105:0/1280668109 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc009690 con 0x7fc4ec003b40 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.005+0000 7fc508457640 1 -- 172.21.15.105:0/1280668109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4ec003b40 msgr2=0x7fc4ec003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.005+0000 7fc508457640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4ec003b40 0x7fc4ec003f40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009ac0 tx=0x7fc4ec005830 comp rx=0 tx=0).stop 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.006+0000 7fc508457640 1 -- 172.21.15.105:0/1280668109 shutdown_connections 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.006+0000 7fc508457640 1 --2- 172.21.15.105:0/1280668109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4ec003b40 0x7fc4ec003f40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.006+0000 7fc508457640 1 -- 172.21.15.105:0/1280668109 >> 172.21.15.105:0/1280668109 conn(0x7fc5000b1740 msgr2=0x7fc5000b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.006+0000 7fc508457640 1 -- 172.21.15.105:0/1280668109 shutdown_connections 2024-03-08T08:28:34.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.006+0000 7fc508457640 1 -- 172.21.15.105:0/1280668109 wait complete. 2024-03-08T08:28:34.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.007+0000 7fc508457640 1 Processor -- start 2024-03-08T08:28:34.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.007+0000 7fc508457640 1 -- start start 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.007+0000 7fc508457640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec003b40 0x7fc5001f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.007+0000 7fc508457640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc50014b670 0x7fc5000c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.007+0000 7fc508457640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc500150d80 0x7fc500143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.007+0000 7fc508457640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5000729a0 con 0x7fc50014b670 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.007+0000 7fc508457640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc500072820 con 0x7fc500150d80 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.007+0000 7fc508457640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50006e510 con 0x7fc4ec003b40 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc50014b670 0x7fc5000c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc50014b670 0x7fc5000c9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36568/0 (socket says 172.21.15.105:36568) 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 -- 172.21.15.105:0/1271776323 learned_addr learned my addr 172.21.15.105:0/1271776323 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5069f4640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc500150d80 0x7fc500143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec003b40 0x7fc5001f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 -- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec003b40 msgr2=0x7fc5001f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec003b40 0x7fc5001f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 -- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc500150d80 msgr2=0x7fc500143be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc500150d80 0x7fc500143be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 -- 172.21.15.105:0/1271776323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5000cb1a0 con 0x7fc50014b670 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec003b40 0x7fc5001f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc5059f2640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc50014b670 0x7fc5000c9240 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0007ae0 tx=0x7fc4f000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc4f6ffd640 1 -- 172.21.15.105:0/1271776323 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00052e0 con 0x7fc50014b670 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc4f6ffd640 1 -- 172.21.15.105:0/1271776323 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f0018070 con 0x7fc50014b670 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5000cb420 con 0x7fc50014b670 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc4f6ffd640 1 -- 172.21.15.105:0/1271776323 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0003830 con 0x7fc50014b670 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.008+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50014f250 con 0x7fc50014b670 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.009+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5000cb940 con 0x7fc50014b670 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.010+0000 7fc4f6ffd640 1 -- 172.21.15.105:0/1271776323 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4f00039d0 con 0x7fc50014b670 2024-03-08T08:28:34.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.010+0000 7fc4f6ffd640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4e406b500 0x7fc4e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.010+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4e406b500 0x7fc4e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.010+0000 7fc4f6ffd640 1 -- 172.21.15.105:0/1271776323 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4f0089e10 con 0x7fc50014b670 2024-03-08T08:28:34.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.011+0000 7fc5061f3640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4e406b500 0x7fc4e406d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc036000 tx=0x7fc4fc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.015+0000 7fc4f6ffd640 1 -- 172.21.15.105:0/1271776323 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5000cb940 con 0x7fc50014b670 2024-03-08T08:28:34.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.221+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc500075120 con 0x7fc50014b670 2024-03-08T08:28:34.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.222+0000 7fc4f6ffd640 1 -- 172.21.15.105:0/1271776323 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fc5000cb1a0 con 0x7fc50014b670 2024-03-08T08:28:34.224 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:34.224 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2759286099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2759286099},{"type":"v1","addr":"172.21.15.144:6834","nonce":2759286099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2788852991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2788852991},{"type":"v1","addr":"172.21.15.144:6835","nonce":2788852991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2226664928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2226664928},{"type":"v1","addr":"172.21.15.105:6835","nonce":2226664928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:28:31.203118+0000","modified":"2024-03-08T08:28:33.213155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2431299564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2431299564},{"type":"v1","addr":"172.21.15.105:6837","nonce":2431299564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T08:28:34.224 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 13 2024-03-08T08:28:34.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4e406b500 msgr2=0x7fc4e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4e406b500 0x7fc4e406d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc036000 tx=0x7fc4fc030000 comp rx=0 tx=0).stop 2024-03-08T08:28:34.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc50014b670 msgr2=0x7fc5000c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc50014b670 0x7fc5000c9240 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0007ae0 tx=0x7fc4f000e960 comp rx=0 tx=0).stop 2024-03-08T08:28:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 shutdown_connections 2024-03-08T08:28:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4e406b500 0x7fc4e406d9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc500150d80 0x7fc500143be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc50014b670 0x7fc5000c9240 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 --2- 172.21.15.105:0/1271776323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4ec003b40 0x7fc5001f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 >> 172.21.15.105:0/1271776323 conn(0x7fc5000b1740 msgr2=0x7fc5000cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 shutdown_connections 2024-03-08T08:28:34.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.224+0000 7fc508457640 1 -- 172.21.15.105:0/1271776323 wait complete. 2024-03-08T08:28:34.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:28:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.335+0000 7fb279fd7640 1 Processor -- start 2024-03-08T08:28:34.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.335+0000 7fb279fd7640 1 -- start start 2024-03-08T08:28:34.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb279fd7640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2740d43e0 0x7fb2740b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb279fd7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 0x7fb27415de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb279fd7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2740d48f0 con 0x7fb2740d2530 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb279fd7640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2740d4bb0 con 0x7fb2740d43e0 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb279fd7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740ceb30 con 0x7fb2740d2910 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2737fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb2740d2530 0x7fb2740b3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38054/0 (socket says 172.21.15.105:38054) 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2737fe640 1 -- 172.21.15.105:0/3117030503 learned_addr learned my addr 172.21.15.105:0/3117030503 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb273fff640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 0x7fb27415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293485823 0 0) 0x7fb2740d48f0 con 0x7fb2740d2530 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb272ffd640 1 --2- 172.21.15.105:0/3117030503 >> v2:172.21.15.144:3300/0 conn(0x7fb2740d43e0 0x7fb2740b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb25c003520 con 0x7fb2740d2530 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223033935 0 0) 0x7fb25c003520 con 0x7fb2740d2530 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 msgr2=0x7fb27415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 0x7fb27415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 >> v2:172.21.15.144:3300/0 conn(0x7fb2740d43e0 msgr2=0x7fb2740b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 --2- 172.21.15.105:0/3117030503 >> v2:172.21.15.144:3300/0 conn(0x7fb2740d43e0 0x7fb2740b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740cee10 con 0x7fb2740d2530 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fb264001670 con 0x7fb2740d2530 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 >> v1:172.21.15.105:6789/0 conn(0x7fb2740d2530 legacy=0x7fb2740b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 0x7fb25c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb25c003f30 0x7fb25c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb25c004d40 0x7fb25c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740d4bb0 con 0x7fb25c004d40 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.336+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740ceb30 con 0x7fb25c003f30 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00ba10 con 0x7fb2740d2910 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb273fff640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb25c004d40 0x7fb25c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb272ffd640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb25c003f30 0x7fb25c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb2737fe640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 0x7fb25c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb273fff640 1 -- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 msgr2=0x7fb25c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb273fff640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 0x7fb25c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb273fff640 1 -- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb25c003f30 msgr2=0x7fb25c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb273fff640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb25c003f30 0x7fb25c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb273fff640 1 -- 172.21.15.105:0/3117030503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c003520 con 0x7fb25c004d40 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.337+0000 7fb2737fe640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740d2910 0x7fb25c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb273fff640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb25c004d40 0x7fb25c0051c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00e170 tx=0x7fb25c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a886b1f3a401c4f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c039070 con 0x7fb25c004d40 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c02cbf0 con 0x7fb25c004d40 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb2727fc640 1 -- 172.21.15.105:0/3117030503 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c009c10 con 0x7fb25c004d40 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb279fd7640 1 -- 172.21.15.105:0/3117030503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb25c004d40 msgr2=0x7fb25c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb279fd7640 1 --2- 172.21.15.105:0/3117030503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb25c004d40 0x7fb25c0051c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00e170 tx=0x7fb25c005830 comp rx=0 tx=0).stop 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb273fff640 1 -- 172.21.15.105:0/3117030503 reap_dead start 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb279fd7640 1 -- 172.21.15.105:0/3117030503 shutdown_connections 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb279fd7640 1 -- 172.21.15.105:0/3117030503 >> 172.21.15.105:0/3117030503 conn(0x7fb27409cfb0 msgr2=0x7fb2740d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb279fd7640 1 -- 172.21.15.105:0/3117030503 shutdown_connections 2024-03-08T08:28:34.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.338+0000 7fb279fd7640 1 -- 172.21.15.105:0/3117030503 wait complete. 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.339+0000 7fb279fd7640 1 Processor -- start 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb279fd7640 1 -- start start 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb279fd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274157af0 0x7fb2741f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb279fd7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb274157ed0 0x7fb27415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb279fd7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2741f7e00 0x7fb27415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb279fd7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27415de00 con 0x7fb274157ed0 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb279fd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2740b3080 con 0x7fb274157af0 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb279fd7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27415df80 con 0x7fb2741f7e00 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb272ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb274157ed0 0x7fb27415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb272ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb274157ed0 0x7fb27415c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36584/0 (socket says 172.21.15.105:36584) 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.340+0000 7fb272ffd640 1 -- 172.21.15.105:0/4181665565 learned_addr learned my addr 172.21.15.105:0/4181665565 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb2737fe640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274157af0 0x7fb2741f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb273fff640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2741f7e00 0x7fb27415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb272ffd640 1 -- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2741f7e00 msgr2=0x7fb27415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb272ffd640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2741f7e00 0x7fb27415cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb272ffd640 1 -- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274157af0 msgr2=0x7fb2741f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb272ffd640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274157af0 0x7fb2741f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb272ffd640 1 -- 172.21.15.105:0/4181665565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2741f82a0 con 0x7fb274157ed0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb2737fe640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274157af0 0x7fb2741f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb272ffd640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb274157ed0 0x7fb27415c580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb268011960 tx=0x7fb268011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb25bfff640 1 -- 172.21.15.105:0/4181665565 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26800faf0 con 0x7fb274157ed0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb25bfff640 1 -- 172.21.15.105:0/4181665565 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26800fc90 con 0x7fb274157ed0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb25bfff640 1 -- 172.21.15.105:0/4181665565 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268014440 con 0x7fb274157ed0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2741f8580 con 0x7fb274157ed0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.341+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb274158490 con 0x7fb274157ed0 2024-03-08T08:28:34.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.342+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2740cab50 con 0x7fb274157ed0 2024-03-08T08:28:34.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.342+0000 7fb25bfff640 1 -- 172.21.15.105:0/4181665565 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb268012040 con 0x7fb274157ed0 2024-03-08T08:28:34.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.343+0000 7fb25bfff640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb26006b4d0 0x7fb26006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.343+0000 7fb25bfff640 1 -- 172.21.15.105:0/4181665565 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb26808a930 con 0x7fb274157ed0 2024-03-08T08:28:34.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.343+0000 7fb2737fe640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb26006b4d0 0x7fb26006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.343+0000 7fb2737fe640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb26006b4d0 0x7fb26006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb264001050 tx=0x7fb264009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.347+0000 7fb25bfff640 1 -- 172.21.15.105:0/4181665565 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2740cab50 con 0x7fb274157ed0 2024-03-08T08:28:34.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.520+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb274075120 con 0x7fb274157ed0 2024-03-08T08:28:34.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.521+0000 7fb25bfff640 1 -- 172.21.15.105:0/4181665565 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16681 (secure 0 0 0) 0x7fb268019030 con 0x7fb274157ed0 2024-03-08T08:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:34.524 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":33,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:32.876978+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:28:29.680862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T08:28:30.179255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb26006b4d0 msgr2=0x7fb26006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb26006b4d0 0x7fb26006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb264001050 tx=0x7fb264009040 comp rx=0 tx=0).stop 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb274157ed0 msgr2=0x7fb27415c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb274157ed0 0x7fb27415c580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb268011960 tx=0x7fb268011e20 comp rx=0 tx=0).stop 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 shutdown_connections 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb26006b4d0 0x7fb26006d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2741f7e00 0x7fb27415cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb274157ed0 0x7fb27415c580 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 --2- 172.21.15.105:0/4181665565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274157af0 0x7fb2741f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.523+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 >> 172.21.15.105:0/4181665565 conn(0x7fb27409cfb0 msgr2=0x7fb27406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:34.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.524+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 shutdown_connections 2024-03-08T08:28:34.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.524+0000 7fb279fd7640 1 -- 172.21.15.105:0/4181665565 wait complete. 2024-03-08T08:28:34.536 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-08T08:28:34.536 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:28:34.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.634+0000 7f985b948640 1 Processor -- start 2024-03-08T08:28:34.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.635+0000 7f985b948640 1 -- start start 2024-03-08T08:28:34.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.635+0000 7f985b948640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f985406bcb0 0x7f985406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.635+0000 7f985b948640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9854161850 0x7f98540d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.635+0000 7f985b948640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98540d48a0 con 0x7f9854161470 2024-03-08T08:28:34.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.635+0000 7f985b948640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98540d2ba0 con 0x7f9854161850 2024-03-08T08:28:34.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.635+0000 7f985b948640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98540d4b80 con 0x7f985406bcb0 2024-03-08T08:28:34.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.636+0000 7f9859ee5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9854161850 0x7f98540d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.636+0000 7f9859ee5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9854161850 0x7f98540d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36600/0 (socket says 172.21.15.105:36600) 2024-03-08T08:28:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.636+0000 7f9859ee5640 1 -- 172.21.15.105:0/1922411424 learned_addr learned my addr 172.21.15.105:0/1922411424 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.636+0000 7f9858ee3640 1 --2- 172.21.15.105:0/1922411424 >> v2:172.21.15.144:3300/0 conn(0x7f985406bcb0 0x7f985406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.636+0000 7f9859ee5640 1 -- 172.21.15.105:0/1922411424 >> v2:172.21.15.144:3300/0 conn(0x7f985406bcb0 msgr2=0x7f985406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.636+0000 7f9859ee5640 1 --2- 172.21.15.105:0/1922411424 >> v2:172.21.15.144:3300/0 conn(0x7f985406bcb0 0x7f985406c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.636+0000 7f9859ee5640 1 -- 172.21.15.105:0/1922411424 >> v1:172.21.15.144:6789/0 conn(0x7f9854161470 legacy=0x7f985406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.636+0000 7f9859ee5640 1 -- 172.21.15.105:0/1922411424 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985406c170 con 0x7f9854161850 2024-03-08T08:28:34.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f9859ee5640 1 --2- 172.21.15.105:0/1922411424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161850 0x7f98540d2660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f98540d3c80 tx=0x7f984c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d202d3742708e370 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f984bfff640 1 -- 172.21.15.105:0/1922411424 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c02ca80 con 0x7f9854161850 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f984bfff640 1 -- 172.21.15.105:0/1922411424 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c02cc20 con 0x7f9854161850 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f984bfff640 1 -- 172.21.15.105:0/1922411424 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c0343f0 con 0x7f9854161850 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f985b948640 1 -- 172.21.15.105:0/1922411424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161850 msgr2=0x7f98540d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f985b948640 1 --2- 172.21.15.105:0/1922411424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161850 0x7f98540d2660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f98540d3c80 tx=0x7f984c02bee0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f985b948640 1 -- 172.21.15.105:0/1922411424 shutdown_connections 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f985b948640 1 --2- 172.21.15.105:0/1922411424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161850 0x7f98540d2660 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f985b948640 1 --2- 172.21.15.105:0/1922411424 >> v2:172.21.15.144:3300/0 conn(0x7f985406bcb0 0x7f985406c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f985b948640 1 -- 172.21.15.105:0/1922411424 >> 172.21.15.105:0/1922411424 conn(0x7f98540d3ce0 msgr2=0x7f98540d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f985b948640 1 -- 172.21.15.105:0/1922411424 shutdown_connections 2024-03-08T08:28:34.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.637+0000 7f985b948640 1 -- 172.21.15.105:0/1922411424 wait complete. 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f985b948640 1 Processor -- start 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f985b948640 1 -- start start 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f985b948640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f985406bcb0 0x7f985415f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f985b948640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161470 0x7f98540cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f985b948640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9854161850 0x7f98540cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f985b948640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985406cf40 con 0x7f9854161470 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f985b948640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985406d370 con 0x7f985406bcb0 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f985b948640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98540cd020 con 0x7f9854161850 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f9858ee3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161470 0x7f98540cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f9858ee3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161470 0x7f98540cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36614/0 (socket says 172.21.15.105:36614) 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f9858ee3640 1 -- 172.21.15.105:0/443244776 learned_addr learned my addr 172.21.15.105:0/443244776 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f98596e4640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f985406bcb0 0x7f985415f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.639+0000 7f9859ee5640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9854161850 0x7f98540cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9858ee3640 1 -- 172.21.15.105:0/443244776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9854161850 msgr2=0x7f98540cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9858ee3640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9854161850 0x7f98540cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9858ee3640 1 -- 172.21.15.105:0/443244776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f985406bcb0 msgr2=0x7f985415f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9858ee3640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f985406bcb0 0x7f985415f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9858ee3640 1 -- 172.21.15.105:0/443244776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98540cd270 con 0x7f9854161470 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9858ee3640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161470 0x7f98540cf800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f985000d7d0 tx=0x7f985000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9849ffb640 1 -- 172.21.15.105:0/443244776 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850004020 con 0x7f9854161470 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9849ffb640 1 -- 172.21.15.105:0/443244776 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98500041e0 con 0x7f9854161470 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98540cd550 con 0x7f9854161470 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f9849ffb640 1 -- 172.21.15.105:0/443244776 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98500043d0 con 0x7f9854161470 2024-03-08T08:28:34.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.640+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985415fda0 con 0x7f9854161470 2024-03-08T08:28:34.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.641+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9854075120 con 0x7f9854161470 2024-03-08T08:28:34.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.641+0000 7f9849ffb640 1 -- 172.21.15.105:0/443244776 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f985001b020 con 0x7f9854161470 2024-03-08T08:28:34.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.642+0000 7f9849ffb640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f983806b500 0x7f983806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.642+0000 7f9849ffb640 1 -- 172.21.15.105:0/443244776 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9850089e00 con 0x7f9854161470 2024-03-08T08:28:34.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.642+0000 7f98596e4640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f983806b500 0x7f983806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.642+0000 7f98596e4640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f983806b500 0x7f983806d9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9844001780 tx=0x7f9844009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.646+0000 7f9849ffb640 1 -- 172.21.15.105:0/443244776 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98500586c0 con 0x7f9854161470 2024-03-08T08:28:34.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.814+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98540b7080 con 0x7f983806b500 2024-03-08T08:28:34.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.820+0000 7f9849ffb640 1 -- 172.21.15.105:0/443244776 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246633 (secure 0 0 0) 0x7f98540b7080 con 0x7f983806b500 2024-03-08T08:28:34.825 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:34.826 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:28:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f983806b500 msgr2=0x7f983806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f983806b500 0x7f983806d9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9844001780 tx=0x7f9844009040 comp rx=0 tx=0).stop 2024-03-08T08:28:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161470 msgr2=0x7f98540cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161470 0x7f98540cf800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f985000d7d0 tx=0x7f985000dc90 comp rx=0 tx=0).stop 2024-03-08T08:28:34.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 shutdown_connections 2024-03-08T08:28:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f983806b500 0x7f983806d9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9854161850 0x7f98540cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9854161470 0x7f98540cf800 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 --2- 172.21.15.105:0/443244776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f985406bcb0 0x7f985415f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 >> 172.21.15.105:0/443244776 conn(0x7f98540d3ce0 msgr2=0x7f985406b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 shutdown_connections 2024-03-08T08:28:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.823+0000 7f985b948640 1 -- 172.21.15.105:0/443244776 wait complete. 2024-03-08T08:28:34.835 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-03-08T08:28:33.106313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220012,"kb_used_data":3832,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754708,"statfs":{"total":773094113280,"available":772868820992,"internally_reserved":0,"allocated":3923968,"data_stored":2223932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000611"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860746+0000","last_change":"2024-03-08T08:28:30.860746+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860746+0000","last_undegraded":"2024-03-08T08:28:30.860746+0000","last_fullsized":"2024-03-08T08:28:30.860746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867282+0000","last_change":"2024-03-08T08:28:30.867282+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867282+0000","last_undegraded":"2024-03-08T08:28:30.867282+0000","last_fullsized":"2024-03-08T08:28:30.867282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866105+0000","last_change":"2024-03-08T08:28:30.866105+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866105+0000","last_undegraded":"2024-03-08T08:28:30.866105+0000","last_fullsized":"2024-03-08T08:28:30.866105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867093+0000","last_change":"2024-03-08T08:28:30.867093+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867093+0000","last_undegraded":"2024-03-08T08:28:30.867093+0000","last_fullsized":"2024-03-08T08:28:30.867093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865930+0000","last_change":"2024-03-08T08:28:30.865930+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865930+0000","last_undegraded":"2024-03-08T08:28:30.865930+0000","last_fullsized":"2024-03-08T08:28:30.865930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865937+0000","last_change":"2024-03-08T08:28:30.865937+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865937+0000","last_undegraded":"2024-03-08T08:28:30.865937+0000","last_fullsized":"2024-03-08T08:28:30.865937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868703+0000","last_change":"2024-03-08T08:28:30.868703+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868703+0000","last_undegraded":"2024-03-08T08:28:30.868703+0000","last_fullsized":"2024-03-08T08:28:30.868703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.874718+0000","last_change":"2024-03-08T08:28:30.874718+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.874718+0000","last_undegraded":"2024-03-08T08:28:30.874718+0000","last_fullsized":"2024-03-08T08:28:30.874718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862649+0000","last_change":"2024-03-08T08:28:30.862649+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862649+0000","last_undegraded":"2024-03-08T08:28:30.862649+0000","last_fullsized":"2024-03-08T08:28:30.862649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864662+0000","last_change":"2024-03-08T08:28:30.864662+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864662+0000","last_undegraded":"2024-03-08T08:28:30.864662+0000","last_fullsized":"2024-03-08T08:28:30.864662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.870446+0000","last_change":"2024-03-08T08:28:30.870446+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.870446+0000","last_undegraded":"2024-03-08T08:28:30.870446+0000","last_fullsized":"2024-03-08T08:28:30.870446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865570+0000","last_change":"2024-03-08T08:28:30.865570+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865570+0000","last_undegraded":"2024-03-08T08:28:30.865570+0000","last_fullsized":"2024-03-08T08:28:30.865570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863391+0000","last_change":"2024-03-08T08:28:30.863391+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863391+0000","last_undegraded":"2024-03-08T08:28:30.863391+0000","last_fullsized":"2024-03-08T08:28:30.863391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865483+0000","last_change":"2024-03-08T08:28:30.865483+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865483+0000","last_undegraded":"2024-03-08T08:28:30.865483+0000","last_fullsized":"2024-03-08T08:28:30.865483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865601+0000","last_change":"2024-03-08T08:28:30.865601+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865601+0000","last_undegraded":"2024-03-08T08:28:30.865601+0000","last_fullsized":"2024-03-08T08:28:30.865601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868891+0000","last_change":"2024-03-08T08:28:30.868891+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868891+0000","last_undegraded":"2024-03-08T08:28:30.868891+0000","last_fullsized":"2024-03-08T08:28:30.868891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868737+0000","last_change":"2024-03-08T08:28:30.868737+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868737+0000","last_undegraded":"2024-03-08T08:28:30.868737+0000","last_fullsized":"2024-03-08T08:28:30.868737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866857+0000","last_change":"2024-03-08T08:28:30.866857+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866857+0000","last_undegraded":"2024-03-08T08:28:30.866857+0000","last_fullsized":"2024-03-08T08:28:30.866857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867802+0000","last_change":"2024-03-08T08:28:30.867802+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867802+0000","last_undegraded":"2024-03-08T08:28:30.867802+0000","last_fullsized":"2024-03-08T08:28:30.867802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864907+0000","last_change":"2024-03-08T08:28:30.864907+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864907+0000","last_undegraded":"2024-03-08T08:28:30.864907+0000","last_fullsized":"2024-03-08T08:28:30.864907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864118+0000","last_change":"2024-03-08T08:28:30.864118+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864118+0000","last_undegraded":"2024-03-08T08:28:30.864118+0000","last_fullsized":"2024-03-08T08:28:30.864118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862637+0000","last_change":"2024-03-08T08:28:30.862637+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862637+0000","last_undegraded":"2024-03-08T08:28:30.862637+0000","last_fullsized":"2024-03-08T08:28:30.862637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863989+0000","last_change":"2024-03-08T08:28:30.863989+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863989+0000","last_undegraded":"2024-03-08T08:28:30.863989+0000","last_fullsized":"2024-03-08T08:28:30.863989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864765+0000","last_change":"2024-03-08T08:28:30.864765+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864765+0000","last_undegraded":"2024-03-08T08:28:30.864765+0000","last_fullsized":"2024-03-08T08:28:30.864765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864158+0000","last_change":"2024-03-08T08:28:30.864158+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864158+0000","last_undegraded":"2024-03-08T08:28:30.864158+0000","last_fullsized":"2024-03-08T08:28:30.864158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866443+0000","last_change":"2024-03-08T08:28:30.866443+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866443+0000","last_undegraded":"2024-03-08T08:28:30.866443+0000","last_fullsized":"2024-03-08T08:28:30.866443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864121+0000","last_change":"2024-03-08T08:28:30.864121+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864121+0000","last_undegraded":"2024-03-08T08:28:30.864121+0000","last_fullsized":"2024-03-08T08:28:30.864121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864738+0000","last_change":"2024-03-08T08:28:30.864738+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864738+0000","last_undegraded":"2024-03-08T08:28:30.864738+0000","last_fullsized":"2024-03-08T08:28:30.864738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863123+0000","last_change":"2024-03-08T08:28:30.863123+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863123+0000","last_undegraded":"2024-03-08T08:28:30.863123+0000","last_fullsized":"2024-03-08T08:28:30.863123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.869203+0000","last_change":"2024-03-08T08:28:30.869203+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.869203+0000","last_undegraded":"2024-03-08T08:28:30.869203+0000","last_fullsized":"2024-03-08T08:28:30.869203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860468+0000","last_change":"2024-03-08T08:28:30.860468+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860468+0000","last_undegraded":"2024-03-08T08:28:30.860468+0000","last_fullsized":"2024-03-08T08:28:30.860468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.874262+0000","last_change":"2024-03-08T08:28:30.874262+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.874262+0000","last_undegraded":"2024-03-08T08:28:30.874262+0000","last_fullsized":"2024-03-08T08:28:30.874262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.870812+0000","last_change":"2024-03-08T08:28:30.870812+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.870812+0000","last_undegraded":"2024-03-08T08:28:30.870812+0000","last_fullsized":"2024-03-08T08:28:30.870812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874668+0000","last_change":"2024-03-08T08:28:30.869323+0000","last_active":"2024-03-08T08:28:30.874668+0000","last_peered":"2024-03-08T08:28:30.874668+0000","last_clean":"2024-03-08T08:28:30.874668+0000","last_became_active":"2024-03-08T08:28:30.869040+0000","last_became_peered":"2024-03-08T08:28:30.869040+0000","last_unstale":"2024-03-08T08:28:30.874668+0000","last_undegraded":"2024-03-08T08:28:30.874668+0000","last_fullsized":"2024-03-08T08:28:30.874668+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860263+0000","last_change":"2024-03-08T08:28:30.860263+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860263+0000","last_undegraded":"2024-03-08T08:28:30.860263+0000","last_fullsized":"2024-03-08T08:28:30.860263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882378+0000","last_change":"2024-03-08T08:28:30.880963+0000","last_active":"2024-03-08T08:28:30.882378+0000","last_peered":"2024-03-08T08:28:30.882378+0000","last_clean":"2024-03-08T08:28:30.882378+0000","last_became_active":"2024-03-08T08:28:30.880613+0000","last_became_peered":"2024-03-08T08:28:30.880613+0000","last_unstale":"2024-03-08T08:28:30.882378+0000","last_undegraded":"2024-03-08T08:28:30.882378+0000","last_fullsized":"2024-03-08T08:28:30.882378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862889+0000","last_change":"2024-03-08T08:28:30.862889+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862889+0000","last_undegraded":"2024-03-08T08:28:30.862889+0000","last_fullsized":"2024-03-08T08:28:30.862889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882179+0000","last_change":"2024-03-08T08:28:30.877050+0000","last_active":"2024-03-08T08:28:30.882179+0000","last_peered":"2024-03-08T08:28:30.882179+0000","last_clean":"2024-03-08T08:28:30.882179+0000","last_became_active":"2024-03-08T08:28:30.876755+0000","last_became_peered":"2024-03-08T08:28:30.876755+0000","last_unstale":"2024-03-08T08:28:30.882179+0000","last_undegraded":"2024-03-08T08:28:30.882179+0000","last_fullsized":"2024-03-08T08:28:30.882179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:02.666304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863108+0000","last_change":"2024-03-08T08:28:30.863108+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863108+0000","last_undegraded":"2024-03-08T08:28:30.863108+0000","last_fullsized":"2024-03-08T08:28:30.863108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.876096+0000","last_change":"2024-03-08T08:28:30.873806+0000","last_active":"2024-03-08T08:28:30.876096+0000","last_peered":"2024-03-08T08:28:30.876096+0000","last_clean":"2024-03-08T08:28:30.876096+0000","last_became_active":"2024-03-08T08:28:30.873361+0000","last_became_peered":"2024-03-08T08:28:30.873361+0000","last_unstale":"2024-03-08T08:28:30.876096+0000","last_undegraded":"2024-03-08T08:28:30.876096+0000","last_fullsized":"2024-03-08T08:28:30.876096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:13:34.320684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868207+0000","last_change":"2024-03-08T08:28:30.868207+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868207+0000","last_undegraded":"2024-03-08T08:28:30.868207+0000","last_fullsized":"2024-03-08T08:28:30.868207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.880408+0000","last_change":"2024-03-08T08:28:30.876030+0000","last_active":"2024-03-08T08:28:30.880408+0000","last_peered":"2024-03-08T08:28:30.880408+0000","last_clean":"2024-03-08T08:28:30.880408+0000","last_became_active":"2024-03-08T08:28:30.875703+0000","last_became_peered":"2024-03-08T08:28:30.875703+0000","last_unstale":"2024-03-08T08:28:30.880408+0000","last_undegraded":"2024-03-08T08:28:30.880408+0000","last_fullsized":"2024-03-08T08:28:30.880408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:56.785346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864832+0000","last_change":"2024-03-08T08:28:30.864832+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864832+0000","last_undegraded":"2024-03-08T08:28:30.864832+0000","last_fullsized":"2024-03-08T08:28:30.864832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874795+0000","last_change":"2024-03-08T08:28:30.873717+0000","last_active":"2024-03-08T08:28:30.874795+0000","last_peered":"2024-03-08T08:28:30.874795+0000","last_clean":"2024-03-08T08:28:30.874795+0000","last_became_active":"2024-03-08T08:28:30.873429+0000","last_became_peered":"2024-03-08T08:28:30.873429+0000","last_unstale":"2024-03-08T08:28:30.874795+0000","last_undegraded":"2024-03-08T08:28:30.874795+0000","last_fullsized":"2024-03-08T08:28:30.874795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:58.674770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860785+0000","last_change":"2024-03-08T08:28:30.860785+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860785+0000","last_undegraded":"2024-03-08T08:28:30.860785+0000","last_fullsized":"2024-03-08T08:28:30.860785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882264+0000","last_change":"2024-03-08T08:28:30.877804+0000","last_active":"2024-03-08T08:28:30.882264+0000","last_peered":"2024-03-08T08:28:30.882264+0000","last_clean":"2024-03-08T08:28:30.882264+0000","last_became_active":"2024-03-08T08:28:30.877522+0000","last_became_peered":"2024-03-08T08:28:30.877522+0000","last_unstale":"2024-03-08T08:28:30.882264+0000","last_undegraded":"2024-03-08T08:28:30.882264+0000","last_fullsized":"2024-03-08T08:28:30.882264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:39:07.612726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863736+0000","last_change":"2024-03-08T08:28:30.863736+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863736+0000","last_undegraded":"2024-03-08T08:28:30.863736+0000","last_fullsized":"2024-03-08T08:28:30.863736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874386+0000","last_change":"2024-03-08T08:28:30.869588+0000","last_active":"2024-03-08T08:28:30.874386+0000","last_peered":"2024-03-08T08:28:30.874386+0000","last_clean":"2024-03-08T08:28:30.874386+0000","last_became_active":"2024-03-08T08:28:30.869274+0000","last_became_peered":"2024-03-08T08:28:30.869274+0000","last_unstale":"2024-03-08T08:28:30.874386+0000","last_undegraded":"2024-03-08T08:28:30.874386+0000","last_fullsized":"2024-03-08T08:28:30.874386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:59.129966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864881+0000","last_change":"2024-03-08T08:28:30.864881+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864881+0000","last_undegraded":"2024-03-08T08:28:30.864881+0000","last_fullsized":"2024-03-08T08:28:30.864881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884799+0000","last_change":"2024-03-08T08:28:30.881215+0000","last_active":"2024-03-08T08:28:30.884799+0000","last_peered":"2024-03-08T08:28:30.884799+0000","last_clean":"2024-03-08T08:28:30.884799+0000","last_became_active":"2024-03-08T08:28:30.880940+0000","last_became_peered":"2024-03-08T08:28:30.880940+0000","last_unstale":"2024-03-08T08:28:30.884799+0000","last_undegraded":"2024-03-08T08:28:30.884799+0000","last_fullsized":"2024-03-08T08:28:30.884799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866510+0000","last_change":"2024-03-08T08:28:30.866510+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866510+0000","last_undegraded":"2024-03-08T08:28:30.866510+0000","last_fullsized":"2024-03-08T08:28:30.866510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.881459+0000","last_change":"2024-03-08T08:28:30.876384+0000","last_active":"2024-03-08T08:28:30.881459+0000","last_peered":"2024-03-08T08:28:30.881459+0000","last_clean":"2024-03-08T08:28:30.881459+0000","last_became_active":"2024-03-08T08:28:30.876053+0000","last_became_peered":"2024-03-08T08:28:30.876053+0000","last_unstale":"2024-03-08T08:28:30.881459+0000","last_undegraded":"2024-03-08T08:28:30.881459+0000","last_fullsized":"2024-03-08T08:28:30.881459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867480+0000","last_change":"2024-03-08T08:28:30.867480+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867480+0000","last_undegraded":"2024-03-08T08:28:30.867480+0000","last_fullsized":"2024-03-08T08:28:30.867480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.885692+0000","last_change":"2024-03-08T08:28:30.880920+0000","last_active":"2024-03-08T08:28:30.885692+0000","last_peered":"2024-03-08T08:28:30.885692+0000","last_clean":"2024-03-08T08:28:30.885692+0000","last_became_active":"2024-03-08T08:28:30.880573+0000","last_became_peered":"2024-03-08T08:28:30.880573+0000","last_unstale":"2024-03-08T08:28:30.885692+0000","last_undegraded":"2024-03-08T08:28:30.885692+0000","last_fullsized":"2024-03-08T08:28:30.885692+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866552+0000","last_change":"2024-03-08T08:28:30.866552+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866552+0000","last_undegraded":"2024-03-08T08:28:30.866552+0000","last_fullsized":"2024-03-08T08:28:30.866552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884768+0000","last_change":"2024-03-08T08:28:30.879905+0000","last_active":"2024-03-08T08:28:30.884768+0000","last_peered":"2024-03-08T08:28:30.884768+0000","last_clean":"2024-03-08T08:28:30.884768+0000","last_became_active":"2024-03-08T08:28:30.879646+0000","last_became_peered":"2024-03-08T08:28:30.879646+0000","last_unstale":"2024-03-08T08:28:30.884768+0000","last_undegraded":"2024-03-08T08:28:30.884768+0000","last_fullsized":"2024-03-08T08:28:30.884768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863054+0000","last_change":"2024-03-08T08:28:30.863054+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863054+0000","last_undegraded":"2024-03-08T08:28:30.863054+0000","last_fullsized":"2024-03-08T08:28:30.863054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874360+0000","last_change":"2024-03-08T08:28:30.869336+0000","last_active":"2024-03-08T08:28:30.874360+0000","last_peered":"2024-03-08T08:28:30.874360+0000","last_clean":"2024-03-08T08:28:30.874360+0000","last_became_active":"2024-03-08T08:28:30.868819+0000","last_became_peered":"2024-03-08T08:28:30.868819+0000","last_unstale":"2024-03-08T08:28:30.874360+0000","last_undegraded":"2024-03-08T08:28:30.874360+0000","last_fullsized":"2024-03-08T08:28:30.874360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:04:25.836412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867856+0000","last_change":"2024-03-08T08:28:30.867856+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867856+0000","last_undegraded":"2024-03-08T08:28:30.867856+0000","last_fullsized":"2024-03-08T08:28:30.867856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.872775+0000","last_change":"2024-03-08T08:28:30.866735+0000","last_active":"2024-03-08T08:28:30.872775+0000","last_peered":"2024-03-08T08:28:30.872775+0000","last_clean":"2024-03-08T08:28:30.872775+0000","last_became_active":"2024-03-08T08:28:30.866479+0000","last_became_peered":"2024-03-08T08:28:30.866479+0000","last_unstale":"2024-03-08T08:28:30.872775+0000","last_undegraded":"2024-03-08T08:28:30.872775+0000","last_fullsized":"2024-03-08T08:28:30.872775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:55.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.873789+0000","last_change":"2024-03-08T08:28:30.873789+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.873789+0000","last_undegraded":"2024-03-08T08:28:30.873789+0000","last_fullsized":"2024-03-08T08:28:30.873789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884793+0000","last_change":"2024-03-08T08:28:30.881378+0000","last_active":"2024-03-08T08:28:30.884793+0000","last_peered":"2024-03-08T08:28:30.884793+0000","last_clean":"2024-03-08T08:28:30.884793+0000","last_became_active":"2024-03-08T08:28:30.880959+0000","last_became_peered":"2024-03-08T08:28:30.880959+0000","last_unstale":"2024-03-08T08:28:30.884793+0000","last_undegraded":"2024-03-08T08:28:30.884793+0000","last_fullsized":"2024-03-08T08:28:30.884793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:06:02.644309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.859877+0000","last_change":"2024-03-08T08:28:30.859877+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.859877+0000","last_undegraded":"2024-03-08T08:28:30.859877+0000","last_fullsized":"2024-03-08T08:28:30.859877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.878480+0000","last_change":"2024-03-08T08:28:30.875030+0000","last_active":"2024-03-08T08:28:30.878480+0000","last_peered":"2024-03-08T08:28:30.878480+0000","last_clean":"2024-03-08T08:28:30.878480+0000","last_became_active":"2024-03-08T08:28:30.874753+0000","last_became_peered":"2024-03-08T08:28:30.874753+0000","last_unstale":"2024-03-08T08:28:30.878480+0000","last_undegraded":"2024-03-08T08:28:30.878480+0000","last_fullsized":"2024-03-08T08:28:30.878480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862859+0000","last_change":"2024-03-08T08:28:30.862859+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862859+0000","last_undegraded":"2024-03-08T08:28:30.862859+0000","last_fullsized":"2024-03-08T08:28:30.862859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879326+0000","last_change":"2024-03-08T08:28:30.873585+0000","last_active":"2024-03-08T08:28:30.879326+0000","last_peered":"2024-03-08T08:28:30.879326+0000","last_clean":"2024-03-08T08:28:30.879326+0000","last_became_active":"2024-03-08T08:28:30.873290+0000","last_became_peered":"2024-03-08T08:28:30.873290+0000","last_unstale":"2024-03-08T08:28:30.879326+0000","last_undegraded":"2024-03-08T08:28:30.879326+0000","last_fullsized":"2024-03-08T08:28:30.879326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:25:07.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866610+0000","last_change":"2024-03-08T08:28:30.866610+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866610+0000","last_undegraded":"2024-03-08T08:28:30.866610+0000","last_fullsized":"2024-03-08T08:28:30.866610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874523+0000","last_change":"2024-03-08T08:28:30.871544+0000","last_active":"2024-03-08T08:28:30.874523+0000","last_peered":"2024-03-08T08:28:30.874523+0000","last_clean":"2024-03-08T08:28:30.874523+0000","last_became_active":"2024-03-08T08:28:30.871236+0000","last_became_peered":"2024-03-08T08:28:30.871236+0000","last_unstale":"2024-03-08T08:28:30.874523+0000","last_undegraded":"2024-03-08T08:28:30.874523+0000","last_fullsized":"2024-03-08T08:28:30.874523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:51.840800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862334+0000","last_change":"2024-03-08T08:28:30.862334+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862334+0000","last_undegraded":"2024-03-08T08:28:30.862334+0000","last_fullsized":"2024-03-08T08:28:30.862334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874432+0000","last_change":"2024-03-08T08:28:30.869691+0000","last_active":"2024-03-08T08:28:30.874432+0000","last_peered":"2024-03-08T08:28:30.874432+0000","last_clean":"2024-03-08T08:28:30.874432+0000","last_became_active":"2024-03-08T08:28:30.869367+0000","last_became_peered":"2024-03-08T08:28:30.869367+0000","last_unstale":"2024-03-08T08:28:30.874432+0000","last_undegraded":"2024-03-08T08:28:30.874432+0000","last_fullsized":"2024-03-08T08:28:30.874432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:22:49.859306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865868+0000","last_change":"2024-03-08T08:28:30.865868+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865868+0000","last_undegraded":"2024-03-08T08:28:30.865868+0000","last_fullsized":"2024-03-08T08:28:30.865868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879893+0000","last_change":"2024-03-08T08:28:30.874068+0000","last_active":"2024-03-08T08:28:30.879893+0000","last_peered":"2024-03-08T08:28:30.879893+0000","last_clean":"2024-03-08T08:28:30.879893+0000","last_became_active":"2024-03-08T08:28:30.873748+0000","last_became_peered":"2024-03-08T08:28:30.873748+0000","last_unstale":"2024-03-08T08:28:30.879893+0000","last_undegraded":"2024-03-08T08:28:30.879893+0000","last_fullsized":"2024-03-08T08:28:30.879893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:13:34.320684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862899+0000","last_change":"2024-03-08T08:28:30.862899+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862899+0000","last_undegraded":"2024-03-08T08:28:30.862899+0000","last_fullsized":"2024-03-08T08:28:30.862899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882325+0000","last_change":"2024-03-08T08:28:30.880129+0000","last_active":"2024-03-08T08:28:30.882325+0000","last_peered":"2024-03-08T08:28:30.882325+0000","last_clean":"2024-03-08T08:28:30.882325+0000","last_became_active":"2024-03-08T08:28:30.879847+0000","last_became_peered":"2024-03-08T08:28:30.879847+0000","last_unstale":"2024-03-08T08:28:30.882325+0000","last_undegraded":"2024-03-08T08:28:30.882325+0000","last_fullsized":"2024-03-08T08:28:30.882325+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862815+0000","last_change":"2024-03-08T08:28:30.862815+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862815+0000","last_undegraded":"2024-03-08T08:28:30.862815+0000","last_fullsized":"2024-03-08T08:28:30.862815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.868560+0000","last_change":"2024-03-08T08:28:30.866833+0000","last_active":"2024-03-08T08:28:30.868560+0000","last_peered":"2024-03-08T08:28:30.868560+0000","last_clean":"2024-03-08T08:28:30.868560+0000","last_became_active":"2024-03-08T08:28:30.866548+0000","last_became_peered":"2024-03-08T08:28:30.866548+0000","last_unstale":"2024-03-08T08:28:30.868560+0000","last_undegraded":"2024-03-08T08:28:30.868560+0000","last_fullsized":"2024-03-08T08:28:30.868560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863027+0000","last_change":"2024-03-08T08:28:30.863027+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863027+0000","last_undegraded":"2024-03-08T08:28:30.863027+0000","last_fullsized":"2024-03-08T08:28:30.863027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874381+0000","last_change":"2024-03-08T08:28:30.869452+0000","last_active":"2024-03-08T08:28:30.874381+0000","last_peered":"2024-03-08T08:28:30.874381+0000","last_clean":"2024-03-08T08:28:30.874381+0000","last_became_active":"2024-03-08T08:28:30.868730+0000","last_became_peered":"2024-03-08T08:28:30.868730+0000","last_unstale":"2024-03-08T08:28:30.874381+0000","last_undegraded":"2024-03-08T08:28:30.874381+0000","last_fullsized":"2024-03-08T08:28:30.874381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:37:44.211703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.869141+0000","last_change":"2024-03-08T08:28:30.869141+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.869141+0000","last_undegraded":"2024-03-08T08:28:30.869141+0000","last_fullsized":"2024-03-08T08:28:30.869141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.887238+0000","last_change":"2024-03-08T08:28:30.881897+0000","last_active":"2024-03-08T08:28:30.887238+0000","last_peered":"2024-03-08T08:28:30.887238+0000","last_clean":"2024-03-08T08:28:30.887238+0000","last_became_active":"2024-03-08T08:28:30.881553+0000","last_became_peered":"2024-03-08T08:28:30.881553+0000","last_unstale":"2024-03-08T08:28:30.887238+0000","last_undegraded":"2024-03-08T08:28:30.887238+0000","last_fullsized":"2024-03-08T08:28:30.887238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864018+0000","last_change":"2024-03-08T08:28:30.864018+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864018+0000","last_undegraded":"2024-03-08T08:28:30.864018+0000","last_fullsized":"2024-03-08T08:28:30.864018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.876217+0000","last_change":"2024-03-08T08:28:30.869401+0000","last_active":"2024-03-08T08:28:30.876217+0000","last_peered":"2024-03-08T08:28:30.876217+0000","last_clean":"2024-03-08T08:28:30.876217+0000","last_became_active":"2024-03-08T08:28:30.869130+0000","last_became_peered":"2024-03-08T08:28:30.869130+0000","last_unstale":"2024-03-08T08:28:30.876217+0000","last_undegraded":"2024-03-08T08:28:30.876217+0000","last_fullsized":"2024-03-08T08:28:30.876217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866183+0000","last_change":"2024-03-08T08:28:30.866183+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866183+0000","last_undegraded":"2024-03-08T08:28:30.866183+0000","last_fullsized":"2024-03-08T08:28:30.866183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882345+0000","last_change":"2024-03-08T08:28:30.880219+0000","last_active":"2024-03-08T08:28:30.882345+0000","last_peered":"2024-03-08T08:28:30.882345+0000","last_clean":"2024-03-08T08:28:30.882345+0000","last_became_active":"2024-03-08T08:28:30.879954+0000","last_became_peered":"2024-03-08T08:28:30.879954+0000","last_unstale":"2024-03-08T08:28:30.882345+0000","last_undegraded":"2024-03-08T08:28:30.882345+0000","last_fullsized":"2024-03-08T08:28:30.882345+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864671+0000","last_change":"2024-03-08T08:28:30.864671+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864671+0000","last_undegraded":"2024-03-08T08:28:30.864671+0000","last_fullsized":"2024-03-08T08:28:30.864671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879349+0000","last_change":"2024-03-08T08:28:30.873467+0000","last_active":"2024-03-08T08:28:30.879349+0000","last_peered":"2024-03-08T08:28:30.879349+0000","last_clean":"2024-03-08T08:28:30.879349+0000","last_became_active":"2024-03-08T08:28:30.873184+0000","last_became_peered":"2024-03-08T08:28:30.873184+0000","last_unstale":"2024-03-08T08:28:30.879349+0000","last_undegraded":"2024-03-08T08:28:30.879349+0000","last_fullsized":"2024-03-08T08:28:30.879349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:05:56.384648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860898+0000","last_change":"2024-03-08T08:28:30.860898+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860898+0000","last_undegraded":"2024-03-08T08:28:30.860898+0000","last_fullsized":"2024-03-08T08:28:30.860898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882365+0000","last_change":"2024-03-08T08:28:30.880960+0000","last_active":"2024-03-08T08:28:30.882365+0000","last_peered":"2024-03-08T08:28:30.882365+0000","last_clean":"2024-03-08T08:28:30.882365+0000","last_became_active":"2024-03-08T08:28:30.880615+0000","last_became_peered":"2024-03-08T08:28:30.880615+0000","last_unstale":"2024-03-08T08:28:30.882365+0000","last_undegraded":"2024-03-08T08:28:30.882365+0000","last_fullsized":"2024-03-08T08:28:30.882365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:55.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.871147+0000","last_change":"2024-03-08T08:28:30.871147+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.871147+0000","last_undegraded":"2024-03-08T08:28:30.871147+0000","last_fullsized":"2024-03-08T08:28:30.871147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879388+0000","last_change":"2024-03-08T08:28:30.873910+0000","last_active":"2024-03-08T08:28:30.879388+0000","last_peered":"2024-03-08T08:28:30.879388+0000","last_clean":"2024-03-08T08:28:30.879388+0000","last_became_active":"2024-03-08T08:28:30.872969+0000","last_became_peered":"2024-03-08T08:28:30.872969+0000","last_unstale":"2024-03-08T08:28:30.879388+0000","last_undegraded":"2024-03-08T08:28:30.879388+0000","last_fullsized":"2024-03-08T08:28:30.879388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864159+0000","last_change":"2024-03-08T08:28:30.864159+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864159+0000","last_undegraded":"2024-03-08T08:28:30.864159+0000","last_fullsized":"2024-03-08T08:28:30.864159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874313+0000","last_change":"2024-03-08T08:28:30.869083+0000","last_active":"2024-03-08T08:28:30.874313+0000","last_peered":"2024-03-08T08:28:30.874313+0000","last_clean":"2024-03-08T08:28:30.874313+0000","last_became_active":"2024-03-08T08:28:30.868810+0000","last_became_peered":"2024-03-08T08:28:30.868810+0000","last_unstale":"2024-03-08T08:28:30.874313+0000","last_undegraded":"2024-03-08T08:28:30.874313+0000","last_fullsized":"2024-03-08T08:28:30.874313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:56.443335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862145+0000","last_change":"2024-03-08T08:28:30.862145+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862145+0000","last_undegraded":"2024-03-08T08:28:30.862145+0000","last_fullsized":"2024-03-08T08:28:30.862145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884834+0000","last_change":"2024-03-08T08:28:30.881900+0000","last_active":"2024-03-08T08:28:30.884834+0000","last_peered":"2024-03-08T08:28:30.884834+0000","last_clean":"2024-03-08T08:28:30.884834+0000","last_became_active":"2024-03-08T08:28:30.881645+0000","last_became_peered":"2024-03-08T08:28:30.881645+0000","last_unstale":"2024-03-08T08:28:30.884834+0000","last_undegraded":"2024-03-08T08:28:30.884834+0000","last_fullsized":"2024-03-08T08:28:30.884834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:48:18.245357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868326+0000","last_change":"2024-03-08T08:28:30.868326+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868326+0000","last_undegraded":"2024-03-08T08:28:30.868326+0000","last_fullsized":"2024-03-08T08:28:30.868326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879950+0000","last_change":"2024-03-08T08:28:30.873407+0000","last_active":"2024-03-08T08:28:30.879950+0000","last_peered":"2024-03-08T08:28:30.879950+0000","last_clean":"2024-03-08T08:28:30.879950+0000","last_became_active":"2024-03-08T08:28:30.871776+0000","last_became_peered":"2024-03-08T08:28:30.871776+0000","last_unstale":"2024-03-08T08:28:30.879950+0000","last_undegraded":"2024-03-08T08:28:30.879950+0000","last_fullsized":"2024-03-08T08:28:30.879950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874963+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:28:30.874963+0000","last_peered":"2024-03-08T08:28:30.874963+0000","last_clean":"2024-03-08T08:28:30.874963+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:28:30.874963+0000","last_undegraded":"2024-03-08T08:28:30.874963+0000","last_fullsized":"2024-03-08T08:28:30.874963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:28:34.839 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:28:34.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.974+0000 7f4ea5de9640 1 Processor -- start 2024-03-08T08:28:34.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.974+0000 7f4ea5de9640 1 -- start start 2024-03-08T08:28:34.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4ea5de9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b4d50 0x7f4ea015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4ea5de9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea00af8b0 con 0x7f4ea01584b0 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4ea5de9640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea00ad910 con 0x7f4ea00b2e80 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4ea5de9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00adbf0 con 0x7f4ea00b4d50 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4e9effd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4ea01584b0 0x7f4ea006cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38070/0 (socket says 172.21.15.105:38070) 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4e9effd640 1 -- 172.21.15.105:0/3476670762 learned_addr learned my addr 172.21.15.105:0/3476670762 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4e9ffff640 1 --2- 172.21.15.105:0/3476670762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b4d50 0x7f4ea015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4e9e7fc640 1 -- 172.21.15.105:0/3476670762 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738031346 0 0) 0x7f4ea00af8b0 con 0x7f4ea01584b0 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4e9e7fc640 1 -- 172.21.15.105:0/3476670762 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e88003520 con 0x7f4ea01584b0 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4e9e7fc640 1 -- 172.21.15.105:0/3476670762 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283988133 0 0) 0x7f4ea00ad910 con 0x7f4ea00b2e80 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.975+0000 7f4e9e7fc640 1 -- 172.21.15.105:0/3476670762 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea00af8b0 con 0x7f4ea00b2e80 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.976+0000 7f4e9ffff640 1 -- 172.21.15.105:0/3476670762 >> v1:172.21.15.144:6789/0 conn(0x7f4ea00b2e80 legacy=0x7f4ea00b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.976+0000 7f4e9ffff640 1 -- 172.21.15.105:0/3476670762 >> v1:172.21.15.105:6789/0 conn(0x7f4ea01584b0 legacy=0x7f4ea006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.976+0000 7f4e9e7fc640 1 -- 172.21.15.105:0/3476670762 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3612384918 0 0) 0x7f4e88003520 con 0x7f4ea01584b0 2024-03-08T08:28:34.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.976+0000 7f4e9ffff640 1 -- 172.21.15.105:0/3476670762 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00aded0 con 0x7f4ea00b4d50 2024-03-08T08:28:34.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.976+0000 7f4e9ffff640 1 --2- 172.21.15.105:0/3476670762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b4d50 0x7f4ea015ece0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009dc0 tx=0x7f4e9802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e3aa556bf9b3b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.976+0000 7f4e9e7fc640 1 -- 172.21.15.105:0/3476670762 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98039070 con 0x7f4ea00b4d50 2024-03-08T08:28:34.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4e9e7fc640 1 -- 172.21.15.105:0/3476670762 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9802c910 con 0x7f4ea00b4d50 2024-03-08T08:28:34.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4e9e7fc640 1 -- 172.21.15.105:0/3476670762 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94001670 con 0x7f4ea00b4d50 2024-03-08T08:28:34.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4ea5de9640 1 -- 172.21.15.105:0/3476670762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b4d50 msgr2=0x7f4ea015ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4ea5de9640 1 --2- 172.21.15.105:0/3476670762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b4d50 0x7f4ea015ece0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009dc0 tx=0x7f4e9802bde0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4ea5de9640 1 -- 172.21.15.105:0/3476670762 shutdown_connections 2024-03-08T08:28:34.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4ea5de9640 1 --2- 172.21.15.105:0/3476670762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b4d50 0x7f4ea015ece0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4ea5de9640 1 -- 172.21.15.105:0/3476670762 >> 172.21.15.105:0/3476670762 conn(0x7f4ea00cfdd0 msgr2=0x7f4ea00d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:34.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4ea5de9640 1 -- 172.21.15.105:0/3476670762 shutdown_connections 2024-03-08T08:28:34.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.977+0000 7f4ea5de9640 1 -- 172.21.15.105:0/3476670762 wait complete. 2024-03-08T08:28:34.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.978+0000 7f4ea5de9640 1 Processor -- start 2024-03-08T08:28:34.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.978+0000 7f4ea5de9640 1 -- start start 2024-03-08T08:28:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4ea5de9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b2e80 0x7f4ea00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4ea5de9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00b4d50 0x7f4ea00cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4ea5de9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea01584b0 0x7f4ea014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4ea5de9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00ae9b0 con 0x7f4ea01584b0 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4ea5de9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00ae490 con 0x7f4ea00b4d50 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4ea5de9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00aeb30 con 0x7f4ea00b2e80 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea01584b0 0x7f4ea014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea01584b0 0x7f4ea014d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36616/0 (socket says 172.21.15.105:36616) 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4e9ffff640 1 -- 172.21.15.105:0/1666127423 learned_addr learned my addr 172.21.15.105:0/1666127423 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4e9effd640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00b4d50 0x7f4ea00cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.979+0000 7f4e9ffff640 1 -- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b2e80 msgr2=0x7f4ea00cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e9f7fe640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b2e80 0x7f4ea00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e9ffff640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b2e80 0x7f4ea00cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e9ffff640 1 -- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00b4d50 msgr2=0x7f4ea00cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e9ffff640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00b4d50 0x7f4ea00cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e9ffff640 1 -- 172.21.15.105:0/1666127423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea01f4180 con 0x7f4ea01584b0 2024-03-08T08:28:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e9effd640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00b4d50 0x7f4ea00cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e9ffff640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea01584b0 0x7f4ea014d630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009ee0 tx=0x7f4e98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e87fff640 1 -- 172.21.15.105:0/1666127423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98039050 con 0x7f4ea01584b0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e87fff640 1 -- 172.21.15.105:0/1666127423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e98045070 con 0x7f4ea01584b0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea01f4460 con 0x7f4ea01584b0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4e87fff640 1 -- 172.21.15.105:0/1666127423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98009450 con 0x7f4ea01584b0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.980+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea01f4d80 con 0x7f4ea01584b0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.981+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea01f4630 con 0x7f4ea01584b0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.981+0000 7f4e87fff640 1 -- 172.21.15.105:0/1666127423 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e98040690 con 0x7f4ea01584b0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.982+0000 7f4e87fff640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e8c06b4d0 0x7f4e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.982+0000 7f4e87fff640 1 -- 172.21.15.105:0/1666127423 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e980aca50 con 0x7f4ea01584b0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.982+0000 7f4e9f7fe640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e8c06b4d0 0x7f4e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:34.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.982+0000 7f4e9f7fe640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e8c06b4d0 0x7f4e8c06d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e900011e0 tx=0x7f4e90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:34.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:34.986+0000 7f4e87fff640 1 -- 172.21.15.105:0/1666127423 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e9807ba20 con 0x7f4ea01584b0 2024-03-08T08:28:35.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.153+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ea01f4860 con 0x7f4e8c06b4d0 2024-03-08T08:28:35.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.159+0000 7f4e87fff640 1 -- 172.21.15.105:0/1666127423 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246633 (secure 0 0 0) 0x7f4e9000d4f0 con 0x7f4e8c06b4d0 2024-03-08T08:28:35.163 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:35.164 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e8c06b4d0 msgr2=0x7f4e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e8c06b4d0 0x7f4e8c06d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e900011e0 tx=0x7f4e90009040 comp rx=0 tx=0).stop 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea01584b0 msgr2=0x7f4ea014d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea01584b0 0x7f4ea014d630 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009ee0 tx=0x7f4e98004a40 comp rx=0 tx=0).stop 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 shutdown_connections 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4e8c06b4d0 0x7f4e8c06d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ea01584b0 0x7f4ea014d630 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00b4d50 0x7f4ea00cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 --2- 172.21.15.105:0/1666127423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ea00b2e80 0x7f4ea00cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 >> 172.21.15.105:0/1666127423 conn(0x7f4ea00cfdd0 msgr2=0x7f4ea015ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:35.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 shutdown_connections 2024-03-08T08:28:35.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.162+0000 7f4ea5de9640 1 -- 172.21.15.105:0/1666127423 wait complete. 2024-03-08T08:28:35.174 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-03-08T08:28:35.106579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220012,"kb_used_data":3832,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754708,"statfs":{"total":773094113280,"available":772868820992,"internally_reserved":0,"allocated":3923968,"data_stored":2223932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.251472"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860746+0000","last_change":"2024-03-08T08:28:30.860746+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860746+0000","last_undegraded":"2024-03-08T08:28:30.860746+0000","last_fullsized":"2024-03-08T08:28:30.860746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867282+0000","last_change":"2024-03-08T08:28:30.867282+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867282+0000","last_undegraded":"2024-03-08T08:28:30.867282+0000","last_fullsized":"2024-03-08T08:28:30.867282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866105+0000","last_change":"2024-03-08T08:28:30.866105+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866105+0000","last_undegraded":"2024-03-08T08:28:30.866105+0000","last_fullsized":"2024-03-08T08:28:30.866105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867093+0000","last_change":"2024-03-08T08:28:30.867093+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867093+0000","last_undegraded":"2024-03-08T08:28:30.867093+0000","last_fullsized":"2024-03-08T08:28:30.867093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865930+0000","last_change":"2024-03-08T08:28:30.865930+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865930+0000","last_undegraded":"2024-03-08T08:28:30.865930+0000","last_fullsized":"2024-03-08T08:28:30.865930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865937+0000","last_change":"2024-03-08T08:28:30.865937+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865937+0000","last_undegraded":"2024-03-08T08:28:30.865937+0000","last_fullsized":"2024-03-08T08:28:30.865937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868703+0000","last_change":"2024-03-08T08:28:30.868703+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868703+0000","last_undegraded":"2024-03-08T08:28:30.868703+0000","last_fullsized":"2024-03-08T08:28:30.868703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.874718+0000","last_change":"2024-03-08T08:28:30.874718+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.874718+0000","last_undegraded":"2024-03-08T08:28:30.874718+0000","last_fullsized":"2024-03-08T08:28:30.874718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862649+0000","last_change":"2024-03-08T08:28:30.862649+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862649+0000","last_undegraded":"2024-03-08T08:28:30.862649+0000","last_fullsized":"2024-03-08T08:28:30.862649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864662+0000","last_change":"2024-03-08T08:28:30.864662+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864662+0000","last_undegraded":"2024-03-08T08:28:30.864662+0000","last_fullsized":"2024-03-08T08:28:30.864662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.870446+0000","last_change":"2024-03-08T08:28:30.870446+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.870446+0000","last_undegraded":"2024-03-08T08:28:30.870446+0000","last_fullsized":"2024-03-08T08:28:30.870446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865570+0000","last_change":"2024-03-08T08:28:30.865570+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865570+0000","last_undegraded":"2024-03-08T08:28:30.865570+0000","last_fullsized":"2024-03-08T08:28:30.865570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863391+0000","last_change":"2024-03-08T08:28:30.863391+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863391+0000","last_undegraded":"2024-03-08T08:28:30.863391+0000","last_fullsized":"2024-03-08T08:28:30.863391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865483+0000","last_change":"2024-03-08T08:28:30.865483+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865483+0000","last_undegraded":"2024-03-08T08:28:30.865483+0000","last_fullsized":"2024-03-08T08:28:30.865483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865601+0000","last_change":"2024-03-08T08:28:30.865601+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865601+0000","last_undegraded":"2024-03-08T08:28:30.865601+0000","last_fullsized":"2024-03-08T08:28:30.865601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868891+0000","last_change":"2024-03-08T08:28:30.868891+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868891+0000","last_undegraded":"2024-03-08T08:28:30.868891+0000","last_fullsized":"2024-03-08T08:28:30.868891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868737+0000","last_change":"2024-03-08T08:28:30.868737+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868737+0000","last_undegraded":"2024-03-08T08:28:30.868737+0000","last_fullsized":"2024-03-08T08:28:30.868737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866857+0000","last_change":"2024-03-08T08:28:30.866857+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866857+0000","last_undegraded":"2024-03-08T08:28:30.866857+0000","last_fullsized":"2024-03-08T08:28:30.866857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867802+0000","last_change":"2024-03-08T08:28:30.867802+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867802+0000","last_undegraded":"2024-03-08T08:28:30.867802+0000","last_fullsized":"2024-03-08T08:28:30.867802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864907+0000","last_change":"2024-03-08T08:28:30.864907+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864907+0000","last_undegraded":"2024-03-08T08:28:30.864907+0000","last_fullsized":"2024-03-08T08:28:30.864907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864118+0000","last_change":"2024-03-08T08:28:30.864118+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864118+0000","last_undegraded":"2024-03-08T08:28:30.864118+0000","last_fullsized":"2024-03-08T08:28:30.864118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862637+0000","last_change":"2024-03-08T08:28:30.862637+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862637+0000","last_undegraded":"2024-03-08T08:28:30.862637+0000","last_fullsized":"2024-03-08T08:28:30.862637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863989+0000","last_change":"2024-03-08T08:28:30.863989+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863989+0000","last_undegraded":"2024-03-08T08:28:30.863989+0000","last_fullsized":"2024-03-08T08:28:30.863989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864765+0000","last_change":"2024-03-08T08:28:30.864765+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864765+0000","last_undegraded":"2024-03-08T08:28:30.864765+0000","last_fullsized":"2024-03-08T08:28:30.864765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864158+0000","last_change":"2024-03-08T08:28:30.864158+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864158+0000","last_undegraded":"2024-03-08T08:28:30.864158+0000","last_fullsized":"2024-03-08T08:28:30.864158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866443+0000","last_change":"2024-03-08T08:28:30.866443+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866443+0000","last_undegraded":"2024-03-08T08:28:30.866443+0000","last_fullsized":"2024-03-08T08:28:30.866443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864121+0000","last_change":"2024-03-08T08:28:30.864121+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864121+0000","last_undegraded":"2024-03-08T08:28:30.864121+0000","last_fullsized":"2024-03-08T08:28:30.864121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864738+0000","last_change":"2024-03-08T08:28:30.864738+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864738+0000","last_undegraded":"2024-03-08T08:28:30.864738+0000","last_fullsized":"2024-03-08T08:28:30.864738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863123+0000","last_change":"2024-03-08T08:28:30.863123+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863123+0000","last_undegraded":"2024-03-08T08:28:30.863123+0000","last_fullsized":"2024-03-08T08:28:30.863123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.869203+0000","last_change":"2024-03-08T08:28:30.869203+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.869203+0000","last_undegraded":"2024-03-08T08:28:30.869203+0000","last_fullsized":"2024-03-08T08:28:30.869203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860468+0000","last_change":"2024-03-08T08:28:30.860468+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860468+0000","last_undegraded":"2024-03-08T08:28:30.860468+0000","last_fullsized":"2024-03-08T08:28:30.860468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.874262+0000","last_change":"2024-03-08T08:28:30.874262+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.874262+0000","last_undegraded":"2024-03-08T08:28:30.874262+0000","last_fullsized":"2024-03-08T08:28:30.874262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.870812+0000","last_change":"2024-03-08T08:28:30.870812+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.870812+0000","last_undegraded":"2024-03-08T08:28:30.870812+0000","last_fullsized":"2024-03-08T08:28:30.870812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874668+0000","last_change":"2024-03-08T08:28:30.869323+0000","last_active":"2024-03-08T08:28:30.874668+0000","last_peered":"2024-03-08T08:28:30.874668+0000","last_clean":"2024-03-08T08:28:30.874668+0000","last_became_active":"2024-03-08T08:28:30.869040+0000","last_became_peered":"2024-03-08T08:28:30.869040+0000","last_unstale":"2024-03-08T08:28:30.874668+0000","last_undegraded":"2024-03-08T08:28:30.874668+0000","last_fullsized":"2024-03-08T08:28:30.874668+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860263+0000","last_change":"2024-03-08T08:28:30.860263+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860263+0000","last_undegraded":"2024-03-08T08:28:30.860263+0000","last_fullsized":"2024-03-08T08:28:30.860263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882378+0000","last_change":"2024-03-08T08:28:30.880963+0000","last_active":"2024-03-08T08:28:30.882378+0000","last_peered":"2024-03-08T08:28:30.882378+0000","last_clean":"2024-03-08T08:28:30.882378+0000","last_became_active":"2024-03-08T08:28:30.880613+0000","last_became_peered":"2024-03-08T08:28:30.880613+0000","last_unstale":"2024-03-08T08:28:30.882378+0000","last_undegraded":"2024-03-08T08:28:30.882378+0000","last_fullsized":"2024-03-08T08:28:30.882378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862889+0000","last_change":"2024-03-08T08:28:30.862889+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862889+0000","last_undegraded":"2024-03-08T08:28:30.862889+0000","last_fullsized":"2024-03-08T08:28:30.862889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882179+0000","last_change":"2024-03-08T08:28:30.877050+0000","last_active":"2024-03-08T08:28:30.882179+0000","last_peered":"2024-03-08T08:28:30.882179+0000","last_clean":"2024-03-08T08:28:30.882179+0000","last_became_active":"2024-03-08T08:28:30.876755+0000","last_became_peered":"2024-03-08T08:28:30.876755+0000","last_unstale":"2024-03-08T08:28:30.882179+0000","last_undegraded":"2024-03-08T08:28:30.882179+0000","last_fullsized":"2024-03-08T08:28:30.882179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:02.666304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863108+0000","last_change":"2024-03-08T08:28:30.863108+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863108+0000","last_undegraded":"2024-03-08T08:28:30.863108+0000","last_fullsized":"2024-03-08T08:28:30.863108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.876096+0000","last_change":"2024-03-08T08:28:30.873806+0000","last_active":"2024-03-08T08:28:30.876096+0000","last_peered":"2024-03-08T08:28:30.876096+0000","last_clean":"2024-03-08T08:28:30.876096+0000","last_became_active":"2024-03-08T08:28:30.873361+0000","last_became_peered":"2024-03-08T08:28:30.873361+0000","last_unstale":"2024-03-08T08:28:30.876096+0000","last_undegraded":"2024-03-08T08:28:30.876096+0000","last_fullsized":"2024-03-08T08:28:30.876096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:13:34.320684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868207+0000","last_change":"2024-03-08T08:28:30.868207+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868207+0000","last_undegraded":"2024-03-08T08:28:30.868207+0000","last_fullsized":"2024-03-08T08:28:30.868207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.880408+0000","last_change":"2024-03-08T08:28:30.876030+0000","last_active":"2024-03-08T08:28:30.880408+0000","last_peered":"2024-03-08T08:28:30.880408+0000","last_clean":"2024-03-08T08:28:30.880408+0000","last_became_active":"2024-03-08T08:28:30.875703+0000","last_became_peered":"2024-03-08T08:28:30.875703+0000","last_unstale":"2024-03-08T08:28:30.880408+0000","last_undegraded":"2024-03-08T08:28:30.880408+0000","last_fullsized":"2024-03-08T08:28:30.880408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:56.785346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864832+0000","last_change":"2024-03-08T08:28:30.864832+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864832+0000","last_undegraded":"2024-03-08T08:28:30.864832+0000","last_fullsized":"2024-03-08T08:28:30.864832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874795+0000","last_change":"2024-03-08T08:28:30.873717+0000","last_active":"2024-03-08T08:28:30.874795+0000","last_peered":"2024-03-08T08:28:30.874795+0000","last_clean":"2024-03-08T08:28:30.874795+0000","last_became_active":"2024-03-08T08:28:30.873429+0000","last_became_peered":"2024-03-08T08:28:30.873429+0000","last_unstale":"2024-03-08T08:28:30.874795+0000","last_undegraded":"2024-03-08T08:28:30.874795+0000","last_fullsized":"2024-03-08T08:28:30.874795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:58.674770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860785+0000","last_change":"2024-03-08T08:28:30.860785+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860785+0000","last_undegraded":"2024-03-08T08:28:30.860785+0000","last_fullsized":"2024-03-08T08:28:30.860785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882264+0000","last_change":"2024-03-08T08:28:30.877804+0000","last_active":"2024-03-08T08:28:30.882264+0000","last_peered":"2024-03-08T08:28:30.882264+0000","last_clean":"2024-03-08T08:28:30.882264+0000","last_became_active":"2024-03-08T08:28:30.877522+0000","last_became_peered":"2024-03-08T08:28:30.877522+0000","last_unstale":"2024-03-08T08:28:30.882264+0000","last_undegraded":"2024-03-08T08:28:30.882264+0000","last_fullsized":"2024-03-08T08:28:30.882264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:39:07.612726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863736+0000","last_change":"2024-03-08T08:28:30.863736+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863736+0000","last_undegraded":"2024-03-08T08:28:30.863736+0000","last_fullsized":"2024-03-08T08:28:30.863736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874386+0000","last_change":"2024-03-08T08:28:30.869588+0000","last_active":"2024-03-08T08:28:30.874386+0000","last_peered":"2024-03-08T08:28:30.874386+0000","last_clean":"2024-03-08T08:28:30.874386+0000","last_became_active":"2024-03-08T08:28:30.869274+0000","last_became_peered":"2024-03-08T08:28:30.869274+0000","last_unstale":"2024-03-08T08:28:30.874386+0000","last_undegraded":"2024-03-08T08:28:30.874386+0000","last_fullsized":"2024-03-08T08:28:30.874386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:59.129966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864881+0000","last_change":"2024-03-08T08:28:30.864881+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864881+0000","last_undegraded":"2024-03-08T08:28:30.864881+0000","last_fullsized":"2024-03-08T08:28:30.864881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884799+0000","last_change":"2024-03-08T08:28:30.881215+0000","last_active":"2024-03-08T08:28:30.884799+0000","last_peered":"2024-03-08T08:28:30.884799+0000","last_clean":"2024-03-08T08:28:30.884799+0000","last_became_active":"2024-03-08T08:28:30.880940+0000","last_became_peered":"2024-03-08T08:28:30.880940+0000","last_unstale":"2024-03-08T08:28:30.884799+0000","last_undegraded":"2024-03-08T08:28:30.884799+0000","last_fullsized":"2024-03-08T08:28:30.884799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866510+0000","last_change":"2024-03-08T08:28:30.866510+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866510+0000","last_undegraded":"2024-03-08T08:28:30.866510+0000","last_fullsized":"2024-03-08T08:28:30.866510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.881459+0000","last_change":"2024-03-08T08:28:30.876384+0000","last_active":"2024-03-08T08:28:30.881459+0000","last_peered":"2024-03-08T08:28:30.881459+0000","last_clean":"2024-03-08T08:28:30.881459+0000","last_became_active":"2024-03-08T08:28:30.876053+0000","last_became_peered":"2024-03-08T08:28:30.876053+0000","last_unstale":"2024-03-08T08:28:30.881459+0000","last_undegraded":"2024-03-08T08:28:30.881459+0000","last_fullsized":"2024-03-08T08:28:30.881459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867480+0000","last_change":"2024-03-08T08:28:30.867480+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867480+0000","last_undegraded":"2024-03-08T08:28:30.867480+0000","last_fullsized":"2024-03-08T08:28:30.867480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.885692+0000","last_change":"2024-03-08T08:28:30.880920+0000","last_active":"2024-03-08T08:28:30.885692+0000","last_peered":"2024-03-08T08:28:30.885692+0000","last_clean":"2024-03-08T08:28:30.885692+0000","last_became_active":"2024-03-08T08:28:30.880573+0000","last_became_peered":"2024-03-08T08:28:30.880573+0000","last_unstale":"2024-03-08T08:28:30.885692+0000","last_undegraded":"2024-03-08T08:28:30.885692+0000","last_fullsized":"2024-03-08T08:28:30.885692+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866552+0000","last_change":"2024-03-08T08:28:30.866552+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866552+0000","last_undegraded":"2024-03-08T08:28:30.866552+0000","last_fullsized":"2024-03-08T08:28:30.866552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884768+0000","last_change":"2024-03-08T08:28:30.879905+0000","last_active":"2024-03-08T08:28:30.884768+0000","last_peered":"2024-03-08T08:28:30.884768+0000","last_clean":"2024-03-08T08:28:30.884768+0000","last_became_active":"2024-03-08T08:28:30.879646+0000","last_became_peered":"2024-03-08T08:28:30.879646+0000","last_unstale":"2024-03-08T08:28:30.884768+0000","last_undegraded":"2024-03-08T08:28:30.884768+0000","last_fullsized":"2024-03-08T08:28:30.884768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863054+0000","last_change":"2024-03-08T08:28:30.863054+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863054+0000","last_undegraded":"2024-03-08T08:28:30.863054+0000","last_fullsized":"2024-03-08T08:28:30.863054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874360+0000","last_change":"2024-03-08T08:28:30.869336+0000","last_active":"2024-03-08T08:28:30.874360+0000","last_peered":"2024-03-08T08:28:30.874360+0000","last_clean":"2024-03-08T08:28:30.874360+0000","last_became_active":"2024-03-08T08:28:30.868819+0000","last_became_peered":"2024-03-08T08:28:30.868819+0000","last_unstale":"2024-03-08T08:28:30.874360+0000","last_undegraded":"2024-03-08T08:28:30.874360+0000","last_fullsized":"2024-03-08T08:28:30.874360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:04:25.836412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867856+0000","last_change":"2024-03-08T08:28:30.867856+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867856+0000","last_undegraded":"2024-03-08T08:28:30.867856+0000","last_fullsized":"2024-03-08T08:28:30.867856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.872775+0000","last_change":"2024-03-08T08:28:30.866735+0000","last_active":"2024-03-08T08:28:30.872775+0000","last_peered":"2024-03-08T08:28:30.872775+0000","last_clean":"2024-03-08T08:28:30.872775+0000","last_became_active":"2024-03-08T08:28:30.866479+0000","last_became_peered":"2024-03-08T08:28:30.866479+0000","last_unstale":"2024-03-08T08:28:30.872775+0000","last_undegraded":"2024-03-08T08:28:30.872775+0000","last_fullsized":"2024-03-08T08:28:30.872775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:55.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.873789+0000","last_change":"2024-03-08T08:28:30.873789+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.873789+0000","last_undegraded":"2024-03-08T08:28:30.873789+0000","last_fullsized":"2024-03-08T08:28:30.873789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884793+0000","last_change":"2024-03-08T08:28:30.881378+0000","last_active":"2024-03-08T08:28:30.884793+0000","last_peered":"2024-03-08T08:28:30.884793+0000","last_clean":"2024-03-08T08:28:30.884793+0000","last_became_active":"2024-03-08T08:28:30.880959+0000","last_became_peered":"2024-03-08T08:28:30.880959+0000","last_unstale":"2024-03-08T08:28:30.884793+0000","last_undegraded":"2024-03-08T08:28:30.884793+0000","last_fullsized":"2024-03-08T08:28:30.884793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:06:02.644309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.859877+0000","last_change":"2024-03-08T08:28:30.859877+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.859877+0000","last_undegraded":"2024-03-08T08:28:30.859877+0000","last_fullsized":"2024-03-08T08:28:30.859877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.878480+0000","last_change":"2024-03-08T08:28:30.875030+0000","last_active":"2024-03-08T08:28:30.878480+0000","last_peered":"2024-03-08T08:28:30.878480+0000","last_clean":"2024-03-08T08:28:30.878480+0000","last_became_active":"2024-03-08T08:28:30.874753+0000","last_became_peered":"2024-03-08T08:28:30.874753+0000","last_unstale":"2024-03-08T08:28:30.878480+0000","last_undegraded":"2024-03-08T08:28:30.878480+0000","last_fullsized":"2024-03-08T08:28:30.878480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862859+0000","last_change":"2024-03-08T08:28:30.862859+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862859+0000","last_undegraded":"2024-03-08T08:28:30.862859+0000","last_fullsized":"2024-03-08T08:28:30.862859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879326+0000","last_change":"2024-03-08T08:28:30.873585+0000","last_active":"2024-03-08T08:28:30.879326+0000","last_peered":"2024-03-08T08:28:30.879326+0000","last_clean":"2024-03-08T08:28:30.879326+0000","last_became_active":"2024-03-08T08:28:30.873290+0000","last_became_peered":"2024-03-08T08:28:30.873290+0000","last_unstale":"2024-03-08T08:28:30.879326+0000","last_undegraded":"2024-03-08T08:28:30.879326+0000","last_fullsized":"2024-03-08T08:28:30.879326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:25:07.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866610+0000","last_change":"2024-03-08T08:28:30.866610+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866610+0000","last_undegraded":"2024-03-08T08:28:30.866610+0000","last_fullsized":"2024-03-08T08:28:30.866610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874523+0000","last_change":"2024-03-08T08:28:30.871544+0000","last_active":"2024-03-08T08:28:30.874523+0000","last_peered":"2024-03-08T08:28:30.874523+0000","last_clean":"2024-03-08T08:28:30.874523+0000","last_became_active":"2024-03-08T08:28:30.871236+0000","last_became_peered":"2024-03-08T08:28:30.871236+0000","last_unstale":"2024-03-08T08:28:30.874523+0000","last_undegraded":"2024-03-08T08:28:30.874523+0000","last_fullsized":"2024-03-08T08:28:30.874523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:51.840800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862334+0000","last_change":"2024-03-08T08:28:30.862334+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862334+0000","last_undegraded":"2024-03-08T08:28:30.862334+0000","last_fullsized":"2024-03-08T08:28:30.862334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874432+0000","last_change":"2024-03-08T08:28:30.869691+0000","last_active":"2024-03-08T08:28:30.874432+0000","last_peered":"2024-03-08T08:28:30.874432+0000","last_clean":"2024-03-08T08:28:30.874432+0000","last_became_active":"2024-03-08T08:28:30.869367+0000","last_became_peered":"2024-03-08T08:28:30.869367+0000","last_unstale":"2024-03-08T08:28:30.874432+0000","last_undegraded":"2024-03-08T08:28:30.874432+0000","last_fullsized":"2024-03-08T08:28:30.874432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:22:49.859306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865868+0000","last_change":"2024-03-08T08:28:30.865868+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865868+0000","last_undegraded":"2024-03-08T08:28:30.865868+0000","last_fullsized":"2024-03-08T08:28:30.865868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879893+0000","last_change":"2024-03-08T08:28:30.874068+0000","last_active":"2024-03-08T08:28:30.879893+0000","last_peered":"2024-03-08T08:28:30.879893+0000","last_clean":"2024-03-08T08:28:30.879893+0000","last_became_active":"2024-03-08T08:28:30.873748+0000","last_became_peered":"2024-03-08T08:28:30.873748+0000","last_unstale":"2024-03-08T08:28:30.879893+0000","last_undegraded":"2024-03-08T08:28:30.879893+0000","last_fullsized":"2024-03-08T08:28:30.879893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:13:34.320684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862899+0000","last_change":"2024-03-08T08:28:30.862899+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862899+0000","last_undegraded":"2024-03-08T08:28:30.862899+0000","last_fullsized":"2024-03-08T08:28:30.862899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882325+0000","last_change":"2024-03-08T08:28:30.880129+0000","last_active":"2024-03-08T08:28:30.882325+0000","last_peered":"2024-03-08T08:28:30.882325+0000","last_clean":"2024-03-08T08:28:30.882325+0000","last_became_active":"2024-03-08T08:28:30.879847+0000","last_became_peered":"2024-03-08T08:28:30.879847+0000","last_unstale":"2024-03-08T08:28:30.882325+0000","last_undegraded":"2024-03-08T08:28:30.882325+0000","last_fullsized":"2024-03-08T08:28:30.882325+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862815+0000","last_change":"2024-03-08T08:28:30.862815+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862815+0000","last_undegraded":"2024-03-08T08:28:30.862815+0000","last_fullsized":"2024-03-08T08:28:30.862815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.868560+0000","last_change":"2024-03-08T08:28:30.866833+0000","last_active":"2024-03-08T08:28:30.868560+0000","last_peered":"2024-03-08T08:28:30.868560+0000","last_clean":"2024-03-08T08:28:30.868560+0000","last_became_active":"2024-03-08T08:28:30.866548+0000","last_became_peered":"2024-03-08T08:28:30.866548+0000","last_unstale":"2024-03-08T08:28:30.868560+0000","last_undegraded":"2024-03-08T08:28:30.868560+0000","last_fullsized":"2024-03-08T08:28:30.868560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863027+0000","last_change":"2024-03-08T08:28:30.863027+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863027+0000","last_undegraded":"2024-03-08T08:28:30.863027+0000","last_fullsized":"2024-03-08T08:28:30.863027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874381+0000","last_change":"2024-03-08T08:28:30.869452+0000","last_active":"2024-03-08T08:28:30.874381+0000","last_peered":"2024-03-08T08:28:30.874381+0000","last_clean":"2024-03-08T08:28:30.874381+0000","last_became_active":"2024-03-08T08:28:30.868730+0000","last_became_peered":"2024-03-08T08:28:30.868730+0000","last_unstale":"2024-03-08T08:28:30.874381+0000","last_undegraded":"2024-03-08T08:28:30.874381+0000","last_fullsized":"2024-03-08T08:28:30.874381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:37:44.211703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.869141+0000","last_change":"2024-03-08T08:28:30.869141+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.869141+0000","last_undegraded":"2024-03-08T08:28:30.869141+0000","last_fullsized":"2024-03-08T08:28:30.869141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.887238+0000","last_change":"2024-03-08T08:28:30.881897+0000","last_active":"2024-03-08T08:28:30.887238+0000","last_peered":"2024-03-08T08:28:30.887238+0000","last_clean":"2024-03-08T08:28:30.887238+0000","last_became_active":"2024-03-08T08:28:30.881553+0000","last_became_peered":"2024-03-08T08:28:30.881553+0000","last_unstale":"2024-03-08T08:28:30.887238+0000","last_undegraded":"2024-03-08T08:28:30.887238+0000","last_fullsized":"2024-03-08T08:28:30.887238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864018+0000","last_change":"2024-03-08T08:28:30.864018+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864018+0000","last_undegraded":"2024-03-08T08:28:30.864018+0000","last_fullsized":"2024-03-08T08:28:30.864018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.876217+0000","last_change":"2024-03-08T08:28:30.869401+0000","last_active":"2024-03-08T08:28:30.876217+0000","last_peered":"2024-03-08T08:28:30.876217+0000","last_clean":"2024-03-08T08:28:30.876217+0000","last_became_active":"2024-03-08T08:28:30.869130+0000","last_became_peered":"2024-03-08T08:28:30.869130+0000","last_unstale":"2024-03-08T08:28:30.876217+0000","last_undegraded":"2024-03-08T08:28:30.876217+0000","last_fullsized":"2024-03-08T08:28:30.876217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866183+0000","last_change":"2024-03-08T08:28:30.866183+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866183+0000","last_undegraded":"2024-03-08T08:28:30.866183+0000","last_fullsized":"2024-03-08T08:28:30.866183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882345+0000","last_change":"2024-03-08T08:28:30.880219+0000","last_active":"2024-03-08T08:28:30.882345+0000","last_peered":"2024-03-08T08:28:30.882345+0000","last_clean":"2024-03-08T08:28:30.882345+0000","last_became_active":"2024-03-08T08:28:30.879954+0000","last_became_peered":"2024-03-08T08:28:30.879954+0000","last_unstale":"2024-03-08T08:28:30.882345+0000","last_undegraded":"2024-03-08T08:28:30.882345+0000","last_fullsized":"2024-03-08T08:28:30.882345+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864671+0000","last_change":"2024-03-08T08:28:30.864671+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864671+0000","last_undegraded":"2024-03-08T08:28:30.864671+0000","last_fullsized":"2024-03-08T08:28:30.864671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879349+0000","last_change":"2024-03-08T08:28:30.873467+0000","last_active":"2024-03-08T08:28:30.879349+0000","last_peered":"2024-03-08T08:28:30.879349+0000","last_clean":"2024-03-08T08:28:30.879349+0000","last_became_active":"2024-03-08T08:28:30.873184+0000","last_became_peered":"2024-03-08T08:28:30.873184+0000","last_unstale":"2024-03-08T08:28:30.879349+0000","last_undegraded":"2024-03-08T08:28:30.879349+0000","last_fullsized":"2024-03-08T08:28:30.879349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:05:56.384648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860898+0000","last_change":"2024-03-08T08:28:30.860898+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860898+0000","last_undegraded":"2024-03-08T08:28:30.860898+0000","last_fullsized":"2024-03-08T08:28:30.860898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882365+0000","last_change":"2024-03-08T08:28:30.880960+0000","last_active":"2024-03-08T08:28:30.882365+0000","last_peered":"2024-03-08T08:28:30.882365+0000","last_clean":"2024-03-08T08:28:30.882365+0000","last_became_active":"2024-03-08T08:28:30.880615+0000","last_became_peered":"2024-03-08T08:28:30.880615+0000","last_unstale":"2024-03-08T08:28:30.882365+0000","last_undegraded":"2024-03-08T08:28:30.882365+0000","last_fullsized":"2024-03-08T08:28:30.882365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:55.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.871147+0000","last_change":"2024-03-08T08:28:30.871147+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.871147+0000","last_undegraded":"2024-03-08T08:28:30.871147+0000","last_fullsized":"2024-03-08T08:28:30.871147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879388+0000","last_change":"2024-03-08T08:28:30.873910+0000","last_active":"2024-03-08T08:28:30.879388+0000","last_peered":"2024-03-08T08:28:30.879388+0000","last_clean":"2024-03-08T08:28:30.879388+0000","last_became_active":"2024-03-08T08:28:30.872969+0000","last_became_peered":"2024-03-08T08:28:30.872969+0000","last_unstale":"2024-03-08T08:28:30.879388+0000","last_undegraded":"2024-03-08T08:28:30.879388+0000","last_fullsized":"2024-03-08T08:28:30.879388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864159+0000","last_change":"2024-03-08T08:28:30.864159+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864159+0000","last_undegraded":"2024-03-08T08:28:30.864159+0000","last_fullsized":"2024-03-08T08:28:30.864159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874313+0000","last_change":"2024-03-08T08:28:30.869083+0000","last_active":"2024-03-08T08:28:30.874313+0000","last_peered":"2024-03-08T08:28:30.874313+0000","last_clean":"2024-03-08T08:28:30.874313+0000","last_became_active":"2024-03-08T08:28:30.868810+0000","last_became_peered":"2024-03-08T08:28:30.868810+0000","last_unstale":"2024-03-08T08:28:30.874313+0000","last_undegraded":"2024-03-08T08:28:30.874313+0000","last_fullsized":"2024-03-08T08:28:30.874313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:56.443335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862145+0000","last_change":"2024-03-08T08:28:30.862145+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862145+0000","last_undegraded":"2024-03-08T08:28:30.862145+0000","last_fullsized":"2024-03-08T08:28:30.862145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884834+0000","last_change":"2024-03-08T08:28:30.881900+0000","last_active":"2024-03-08T08:28:30.884834+0000","last_peered":"2024-03-08T08:28:30.884834+0000","last_clean":"2024-03-08T08:28:30.884834+0000","last_became_active":"2024-03-08T08:28:30.881645+0000","last_became_peered":"2024-03-08T08:28:30.881645+0000","last_unstale":"2024-03-08T08:28:30.884834+0000","last_undegraded":"2024-03-08T08:28:30.884834+0000","last_fullsized":"2024-03-08T08:28:30.884834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:48:18.245357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868326+0000","last_change":"2024-03-08T08:28:30.868326+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868326+0000","last_undegraded":"2024-03-08T08:28:30.868326+0000","last_fullsized":"2024-03-08T08:28:30.868326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879950+0000","last_change":"2024-03-08T08:28:30.873407+0000","last_active":"2024-03-08T08:28:30.879950+0000","last_peered":"2024-03-08T08:28:30.879950+0000","last_clean":"2024-03-08T08:28:30.879950+0000","last_became_active":"2024-03-08T08:28:30.871776+0000","last_became_peered":"2024-03-08T08:28:30.871776+0000","last_unstale":"2024-03-08T08:28:30.879950+0000","last_undegraded":"2024-03-08T08:28:30.879950+0000","last_fullsized":"2024-03-08T08:28:30.879950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874963+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:28:30.874963+0000","last_peered":"2024-03-08T08:28:30.874963+0000","last_clean":"2024-03-08T08:28:30.874963+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:28:30.874963+0000","last_undegraded":"2024-03-08T08:28:30.874963+0000","last_fullsized":"2024-03-08T08:28:30.874963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:28:35.177 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:28:35.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.312+0000 7fb46ec07640 1 Processor -- start 2024-03-08T08:28:35.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.312+0000 7fb46ec07640 1 -- start start 2024-03-08T08:28:35.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb46ec07640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4680b2e80 0x7fb4680b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb46ec07640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb4681584b0 0x7fb46815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb46ec07640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4680af8b0 con 0x7fb4680b3260 2024-03-08T08:28:35.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb46ec07640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4680ad910 con 0x7fb4681584b0 2024-03-08T08:28:35.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb46ec07640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4680adbf0 con 0x7fb4680b2e80 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb467fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb4681584b0 0x7fb46815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb46d1a4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb4680b3260 0x7fb468161850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38074/0 (socket says 172.21.15.105:38074) 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb46d1a4640 1 -- 172.21.15.105:0/3500112601 learned_addr learned my addr 172.21.15.105:0/3500112601 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb46c9a3640 1 --2- 172.21.15.105:0/3500112601 >> v2:172.21.15.144:3300/0 conn(0x7fb4680b2e80 0x7fb4680b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb467fff640 1 -- 172.21.15.105:0/3500112601 >> v2:172.21.15.144:3300/0 conn(0x7fb4680b2e80 msgr2=0x7fb4680b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb467fff640 1 --2- 172.21.15.105:0/3500112601 >> v2:172.21.15.144:3300/0 conn(0x7fb4680b2e80 0x7fb4680b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb467fff640 1 -- 172.21.15.105:0/3500112601 >> v1:172.21.15.105:6789/0 conn(0x7fb4680b3260 legacy=0x7fb468161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb467fff640 1 -- 172.21.15.105:0/3500112601 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680aded0 con 0x7fb4681584b0 2024-03-08T08:28:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.313+0000 7fb4677fe640 1 -- 172.21.15.105:0/3500112601 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132590691 0 0) 0x7fb4680af8b0 con 0x7fb4680b3260 2024-03-08T08:28:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.314+0000 7fb467fff640 1 --2- 172.21.15.105:0/3500112601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4681584b0 0x7fb46815ece0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb46804f200 tx=0x7fb45402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf663d2aa503f84 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.314+0000 7fb4677fe640 1 -- 172.21.15.105:0/3500112601 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454035020 con 0x7fb4681584b0 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.314+0000 7fb4677fe640 1 -- 172.21.15.105:0/3500112601 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454031830 con 0x7fb4681584b0 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.314+0000 7fb4677fe640 1 -- 172.21.15.105:0/3500112601 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454031c00 con 0x7fb4681584b0 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.314+0000 7fb46ec07640 1 -- 172.21.15.105:0/3500112601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4681584b0 msgr2=0x7fb46815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.314+0000 7fb46ec07640 1 --2- 172.21.15.105:0/3500112601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4681584b0 0x7fb46815ece0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb46804f200 tx=0x7fb45402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.315+0000 7fb46ec07640 1 -- 172.21.15.105:0/3500112601 shutdown_connections 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.315+0000 7fb46ec07640 1 --2- 172.21.15.105:0/3500112601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4681584b0 0x7fb46815ece0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.315+0000 7fb46ec07640 1 --2- 172.21.15.105:0/3500112601 >> v2:172.21.15.144:3300/0 conn(0x7fb4680b2e80 0x7fb4680b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.315+0000 7fb46ec07640 1 -- 172.21.15.105:0/3500112601 >> 172.21.15.105:0/3500112601 conn(0x7fb4680cfdd0 msgr2=0x7fb4680d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.315+0000 7fb46ec07640 1 -- 172.21.15.105:0/3500112601 shutdown_connections 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.315+0000 7fb46ec07640 1 -- 172.21.15.105:0/3500112601 wait complete. 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.316+0000 7fb46ec07640 1 Processor -- start 2024-03-08T08:28:35.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.316+0000 7fb46ec07640 1 -- start start 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46ec07640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4680b2e80 0x7fb468159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46ec07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4680b3260 0x7fb46815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46ec07640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4681584b0 0x7fb4680b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46ec07640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680b36b0 con 0x7fb4680b3260 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46ec07640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680ae490 con 0x7fb4680b2e80 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46ec07640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680b3830 con 0x7fb4681584b0 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4680b3260 0x7fb46815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4680b3260 0x7fb46815fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36636/0 (socket says 172.21.15.105:36636) 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 -- 172.21.15.105:0/2857623882 learned_addr learned my addr 172.21.15.105:0/2857623882 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:35.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46c9a3640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4680b2e80 0x7fb468159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 -- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4681584b0 msgr2=0x7fb4680b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46d1a4640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4681584b0 0x7fb4680b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4681584b0 0x7fb4680b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 -- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4680b2e80 msgr2=0x7fb468159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4680b2e80 0x7fb468159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 -- 172.21.15.105:0/2857623882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468158dc0 con 0x7fb4680b3260 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46c9a3640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4680b2e80 0x7fb468159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb467fff640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4680b3260 0x7fb46815fd70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb45402bee0 tx=0x7fb4540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb4657fa640 1 -- 172.21.15.105:0/2857623882 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454041070 con 0x7fb4680b3260 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb4657fa640 1 -- 172.21.15.105:0/2857623882 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45403f070 con 0x7fb4680b3260 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4681590a0 con 0x7fb4680b3260 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb4657fa640 1 -- 172.21.15.105:0/2857623882 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454035040 con 0x7fb4680b3260 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.317+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb468159660 con 0x7fb4680b3260 2024-03-08T08:28:35.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.318+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb468075120 con 0x7fb4680b3260 2024-03-08T08:28:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.319+0000 7fb4657fa640 1 -- 172.21.15.105:0/2857623882 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb454038020 con 0x7fb4680b3260 2024-03-08T08:28:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.319+0000 7fb4657fa640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb44c06b4d0 0x7fb44c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.319+0000 7fb4657fa640 1 -- 172.21.15.105:0/2857623882 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4540ab860 con 0x7fb4680b3260 2024-03-08T08:28:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.319+0000 7fb46c9a3640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb44c06b4d0 0x7fb44c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.319+0000 7fb46c9a3640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb44c06b4d0 0x7fb44c06d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb45c006fd0 tx=0x7fb45c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.323+0000 7fb4657fa640 1 -- 172.21.15.105:0/2857623882 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb45407a770 con 0x7fb4680b3260 2024-03-08T08:28:35.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.490+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4680af250 con 0x7fb44c06b4d0 2024-03-08T08:28:35.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.495+0000 7fb4657fa640 1 -- 172.21.15.105:0/2857623882 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246633 (secure 0 0 0) 0x7fb4680af250 con 0x7fb44c06b4d0 2024-03-08T08:28:35.500 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:35.501 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:28:35.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb44c06b4d0 msgr2=0x7fb44c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb44c06b4d0 0x7fb44c06d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb45c006fd0 tx=0x7fb45c008040 comp rx=0 tx=0).stop 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4680b3260 msgr2=0x7fb46815fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4680b3260 0x7fb46815fd70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb45402bee0 tx=0x7fb4540046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 shutdown_connections 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb44c06b4d0 0x7fb44c06d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4681584b0 0x7fb4680b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4680b3260 0x7fb46815fd70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 --2- 172.21.15.105:0/2857623882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4680b2e80 0x7fb468159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 >> 172.21.15.105:0/2857623882 conn(0x7fb4680cfdd0 msgr2=0x7fb4680cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 shutdown_connections 2024-03-08T08:28:35.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.498+0000 7fb46ec07640 1 -- 172.21.15.105:0/2857623882 wait complete. 2024-03-08T08:28:35.511 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-03-08T08:28:35.106579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220012,"kb_used_data":3832,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754708,"statfs":{"total":773094113280,"available":772868820992,"internally_reserved":0,"allocated":3923968,"data_stored":2223932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.251472"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860746+0000","last_change":"2024-03-08T08:28:30.860746+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860746+0000","last_undegraded":"2024-03-08T08:28:30.860746+0000","last_fullsized":"2024-03-08T08:28:30.860746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867282+0000","last_change":"2024-03-08T08:28:30.867282+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867282+0000","last_undegraded":"2024-03-08T08:28:30.867282+0000","last_fullsized":"2024-03-08T08:28:30.867282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866105+0000","last_change":"2024-03-08T08:28:30.866105+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866105+0000","last_undegraded":"2024-03-08T08:28:30.866105+0000","last_fullsized":"2024-03-08T08:28:30.866105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867093+0000","last_change":"2024-03-08T08:28:30.867093+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867093+0000","last_undegraded":"2024-03-08T08:28:30.867093+0000","last_fullsized":"2024-03-08T08:28:30.867093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865930+0000","last_change":"2024-03-08T08:28:30.865930+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865930+0000","last_undegraded":"2024-03-08T08:28:30.865930+0000","last_fullsized":"2024-03-08T08:28:30.865930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865937+0000","last_change":"2024-03-08T08:28:30.865937+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865937+0000","last_undegraded":"2024-03-08T08:28:30.865937+0000","last_fullsized":"2024-03-08T08:28:30.865937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868703+0000","last_change":"2024-03-08T08:28:30.868703+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868703+0000","last_undegraded":"2024-03-08T08:28:30.868703+0000","last_fullsized":"2024-03-08T08:28:30.868703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.874718+0000","last_change":"2024-03-08T08:28:30.874718+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.874718+0000","last_undegraded":"2024-03-08T08:28:30.874718+0000","last_fullsized":"2024-03-08T08:28:30.874718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862649+0000","last_change":"2024-03-08T08:28:30.862649+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862649+0000","last_undegraded":"2024-03-08T08:28:30.862649+0000","last_fullsized":"2024-03-08T08:28:30.862649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864662+0000","last_change":"2024-03-08T08:28:30.864662+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864662+0000","last_undegraded":"2024-03-08T08:28:30.864662+0000","last_fullsized":"2024-03-08T08:28:30.864662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.870446+0000","last_change":"2024-03-08T08:28:30.870446+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.870446+0000","last_undegraded":"2024-03-08T08:28:30.870446+0000","last_fullsized":"2024-03-08T08:28:30.870446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865570+0000","last_change":"2024-03-08T08:28:30.865570+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865570+0000","last_undegraded":"2024-03-08T08:28:30.865570+0000","last_fullsized":"2024-03-08T08:28:30.865570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863391+0000","last_change":"2024-03-08T08:28:30.863391+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863391+0000","last_undegraded":"2024-03-08T08:28:30.863391+0000","last_fullsized":"2024-03-08T08:28:30.863391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865483+0000","last_change":"2024-03-08T08:28:30.865483+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865483+0000","last_undegraded":"2024-03-08T08:28:30.865483+0000","last_fullsized":"2024-03-08T08:28:30.865483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865601+0000","last_change":"2024-03-08T08:28:30.865601+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865601+0000","last_undegraded":"2024-03-08T08:28:30.865601+0000","last_fullsized":"2024-03-08T08:28:30.865601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868891+0000","last_change":"2024-03-08T08:28:30.868891+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868891+0000","last_undegraded":"2024-03-08T08:28:30.868891+0000","last_fullsized":"2024-03-08T08:28:30.868891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868737+0000","last_change":"2024-03-08T08:28:30.868737+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868737+0000","last_undegraded":"2024-03-08T08:28:30.868737+0000","last_fullsized":"2024-03-08T08:28:30.868737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866857+0000","last_change":"2024-03-08T08:28:30.866857+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866857+0000","last_undegraded":"2024-03-08T08:28:30.866857+0000","last_fullsized":"2024-03-08T08:28:30.866857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867802+0000","last_change":"2024-03-08T08:28:30.867802+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867802+0000","last_undegraded":"2024-03-08T08:28:30.867802+0000","last_fullsized":"2024-03-08T08:28:30.867802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864907+0000","last_change":"2024-03-08T08:28:30.864907+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864907+0000","last_undegraded":"2024-03-08T08:28:30.864907+0000","last_fullsized":"2024-03-08T08:28:30.864907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864118+0000","last_change":"2024-03-08T08:28:30.864118+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864118+0000","last_undegraded":"2024-03-08T08:28:30.864118+0000","last_fullsized":"2024-03-08T08:28:30.864118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862637+0000","last_change":"2024-03-08T08:28:30.862637+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862637+0000","last_undegraded":"2024-03-08T08:28:30.862637+0000","last_fullsized":"2024-03-08T08:28:30.862637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863989+0000","last_change":"2024-03-08T08:28:30.863989+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863989+0000","last_undegraded":"2024-03-08T08:28:30.863989+0000","last_fullsized":"2024-03-08T08:28:30.863989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864765+0000","last_change":"2024-03-08T08:28:30.864765+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864765+0000","last_undegraded":"2024-03-08T08:28:30.864765+0000","last_fullsized":"2024-03-08T08:28:30.864765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864158+0000","last_change":"2024-03-08T08:28:30.864158+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864158+0000","last_undegraded":"2024-03-08T08:28:30.864158+0000","last_fullsized":"2024-03-08T08:28:30.864158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866443+0000","last_change":"2024-03-08T08:28:30.866443+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866443+0000","last_undegraded":"2024-03-08T08:28:30.866443+0000","last_fullsized":"2024-03-08T08:28:30.866443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864121+0000","last_change":"2024-03-08T08:28:30.864121+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864121+0000","last_undegraded":"2024-03-08T08:28:30.864121+0000","last_fullsized":"2024-03-08T08:28:30.864121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864738+0000","last_change":"2024-03-08T08:28:30.864738+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864738+0000","last_undegraded":"2024-03-08T08:28:30.864738+0000","last_fullsized":"2024-03-08T08:28:30.864738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863123+0000","last_change":"2024-03-08T08:28:30.863123+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863123+0000","last_undegraded":"2024-03-08T08:28:30.863123+0000","last_fullsized":"2024-03-08T08:28:30.863123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.869203+0000","last_change":"2024-03-08T08:28:30.869203+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.869203+0000","last_undegraded":"2024-03-08T08:28:30.869203+0000","last_fullsized":"2024-03-08T08:28:30.869203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860468+0000","last_change":"2024-03-08T08:28:30.860468+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860468+0000","last_undegraded":"2024-03-08T08:28:30.860468+0000","last_fullsized":"2024-03-08T08:28:30.860468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.874262+0000","last_change":"2024-03-08T08:28:30.874262+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.874262+0000","last_undegraded":"2024-03-08T08:28:30.874262+0000","last_fullsized":"2024-03-08T08:28:30.874262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.870812+0000","last_change":"2024-03-08T08:28:30.870812+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.870812+0000","last_undegraded":"2024-03-08T08:28:30.870812+0000","last_fullsized":"2024-03-08T08:28:30.870812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874668+0000","last_change":"2024-03-08T08:28:30.869323+0000","last_active":"2024-03-08T08:28:30.874668+0000","last_peered":"2024-03-08T08:28:30.874668+0000","last_clean":"2024-03-08T08:28:30.874668+0000","last_became_active":"2024-03-08T08:28:30.869040+0000","last_became_peered":"2024-03-08T08:28:30.869040+0000","last_unstale":"2024-03-08T08:28:30.874668+0000","last_undegraded":"2024-03-08T08:28:30.874668+0000","last_fullsized":"2024-03-08T08:28:30.874668+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860263+0000","last_change":"2024-03-08T08:28:30.860263+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860263+0000","last_undegraded":"2024-03-08T08:28:30.860263+0000","last_fullsized":"2024-03-08T08:28:30.860263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882378+0000","last_change":"2024-03-08T08:28:30.880963+0000","last_active":"2024-03-08T08:28:30.882378+0000","last_peered":"2024-03-08T08:28:30.882378+0000","last_clean":"2024-03-08T08:28:30.882378+0000","last_became_active":"2024-03-08T08:28:30.880613+0000","last_became_peered":"2024-03-08T08:28:30.880613+0000","last_unstale":"2024-03-08T08:28:30.882378+0000","last_undegraded":"2024-03-08T08:28:30.882378+0000","last_fullsized":"2024-03-08T08:28:30.882378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862889+0000","last_change":"2024-03-08T08:28:30.862889+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862889+0000","last_undegraded":"2024-03-08T08:28:30.862889+0000","last_fullsized":"2024-03-08T08:28:30.862889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882179+0000","last_change":"2024-03-08T08:28:30.877050+0000","last_active":"2024-03-08T08:28:30.882179+0000","last_peered":"2024-03-08T08:28:30.882179+0000","last_clean":"2024-03-08T08:28:30.882179+0000","last_became_active":"2024-03-08T08:28:30.876755+0000","last_became_peered":"2024-03-08T08:28:30.876755+0000","last_unstale":"2024-03-08T08:28:30.882179+0000","last_undegraded":"2024-03-08T08:28:30.882179+0000","last_fullsized":"2024-03-08T08:28:30.882179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:02.666304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863108+0000","last_change":"2024-03-08T08:28:30.863108+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863108+0000","last_undegraded":"2024-03-08T08:28:30.863108+0000","last_fullsized":"2024-03-08T08:28:30.863108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.876096+0000","last_change":"2024-03-08T08:28:30.873806+0000","last_active":"2024-03-08T08:28:30.876096+0000","last_peered":"2024-03-08T08:28:30.876096+0000","last_clean":"2024-03-08T08:28:30.876096+0000","last_became_active":"2024-03-08T08:28:30.873361+0000","last_became_peered":"2024-03-08T08:28:30.873361+0000","last_unstale":"2024-03-08T08:28:30.876096+0000","last_undegraded":"2024-03-08T08:28:30.876096+0000","last_fullsized":"2024-03-08T08:28:30.876096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:13:34.320684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868207+0000","last_change":"2024-03-08T08:28:30.868207+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868207+0000","last_undegraded":"2024-03-08T08:28:30.868207+0000","last_fullsized":"2024-03-08T08:28:30.868207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.880408+0000","last_change":"2024-03-08T08:28:30.876030+0000","last_active":"2024-03-08T08:28:30.880408+0000","last_peered":"2024-03-08T08:28:30.880408+0000","last_clean":"2024-03-08T08:28:30.880408+0000","last_became_active":"2024-03-08T08:28:30.875703+0000","last_became_peered":"2024-03-08T08:28:30.875703+0000","last_unstale":"2024-03-08T08:28:30.880408+0000","last_undegraded":"2024-03-08T08:28:30.880408+0000","last_fullsized":"2024-03-08T08:28:30.880408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:56.785346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864832+0000","last_change":"2024-03-08T08:28:30.864832+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864832+0000","last_undegraded":"2024-03-08T08:28:30.864832+0000","last_fullsized":"2024-03-08T08:28:30.864832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874795+0000","last_change":"2024-03-08T08:28:30.873717+0000","last_active":"2024-03-08T08:28:30.874795+0000","last_peered":"2024-03-08T08:28:30.874795+0000","last_clean":"2024-03-08T08:28:30.874795+0000","last_became_active":"2024-03-08T08:28:30.873429+0000","last_became_peered":"2024-03-08T08:28:30.873429+0000","last_unstale":"2024-03-08T08:28:30.874795+0000","last_undegraded":"2024-03-08T08:28:30.874795+0000","last_fullsized":"2024-03-08T08:28:30.874795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:58.674770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860785+0000","last_change":"2024-03-08T08:28:30.860785+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860785+0000","last_undegraded":"2024-03-08T08:28:30.860785+0000","last_fullsized":"2024-03-08T08:28:30.860785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882264+0000","last_change":"2024-03-08T08:28:30.877804+0000","last_active":"2024-03-08T08:28:30.882264+0000","last_peered":"2024-03-08T08:28:30.882264+0000","last_clean":"2024-03-08T08:28:30.882264+0000","last_became_active":"2024-03-08T08:28:30.877522+0000","last_became_peered":"2024-03-08T08:28:30.877522+0000","last_unstale":"2024-03-08T08:28:30.882264+0000","last_undegraded":"2024-03-08T08:28:30.882264+0000","last_fullsized":"2024-03-08T08:28:30.882264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:39:07.612726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863736+0000","last_change":"2024-03-08T08:28:30.863736+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863736+0000","last_undegraded":"2024-03-08T08:28:30.863736+0000","last_fullsized":"2024-03-08T08:28:30.863736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874386+0000","last_change":"2024-03-08T08:28:30.869588+0000","last_active":"2024-03-08T08:28:30.874386+0000","last_peered":"2024-03-08T08:28:30.874386+0000","last_clean":"2024-03-08T08:28:30.874386+0000","last_became_active":"2024-03-08T08:28:30.869274+0000","last_became_peered":"2024-03-08T08:28:30.869274+0000","last_unstale":"2024-03-08T08:28:30.874386+0000","last_undegraded":"2024-03-08T08:28:30.874386+0000","last_fullsized":"2024-03-08T08:28:30.874386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:59.129966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864881+0000","last_change":"2024-03-08T08:28:30.864881+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864881+0000","last_undegraded":"2024-03-08T08:28:30.864881+0000","last_fullsized":"2024-03-08T08:28:30.864881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884799+0000","last_change":"2024-03-08T08:28:30.881215+0000","last_active":"2024-03-08T08:28:30.884799+0000","last_peered":"2024-03-08T08:28:30.884799+0000","last_clean":"2024-03-08T08:28:30.884799+0000","last_became_active":"2024-03-08T08:28:30.880940+0000","last_became_peered":"2024-03-08T08:28:30.880940+0000","last_unstale":"2024-03-08T08:28:30.884799+0000","last_undegraded":"2024-03-08T08:28:30.884799+0000","last_fullsized":"2024-03-08T08:28:30.884799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866510+0000","last_change":"2024-03-08T08:28:30.866510+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866510+0000","last_undegraded":"2024-03-08T08:28:30.866510+0000","last_fullsized":"2024-03-08T08:28:30.866510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.881459+0000","last_change":"2024-03-08T08:28:30.876384+0000","last_active":"2024-03-08T08:28:30.881459+0000","last_peered":"2024-03-08T08:28:30.881459+0000","last_clean":"2024-03-08T08:28:30.881459+0000","last_became_active":"2024-03-08T08:28:30.876053+0000","last_became_peered":"2024-03-08T08:28:30.876053+0000","last_unstale":"2024-03-08T08:28:30.881459+0000","last_undegraded":"2024-03-08T08:28:30.881459+0000","last_fullsized":"2024-03-08T08:28:30.881459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867480+0000","last_change":"2024-03-08T08:28:30.867480+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867480+0000","last_undegraded":"2024-03-08T08:28:30.867480+0000","last_fullsized":"2024-03-08T08:28:30.867480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.885692+0000","last_change":"2024-03-08T08:28:30.880920+0000","last_active":"2024-03-08T08:28:30.885692+0000","last_peered":"2024-03-08T08:28:30.885692+0000","last_clean":"2024-03-08T08:28:30.885692+0000","last_became_active":"2024-03-08T08:28:30.880573+0000","last_became_peered":"2024-03-08T08:28:30.880573+0000","last_unstale":"2024-03-08T08:28:30.885692+0000","last_undegraded":"2024-03-08T08:28:30.885692+0000","last_fullsized":"2024-03-08T08:28:30.885692+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866552+0000","last_change":"2024-03-08T08:28:30.866552+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866552+0000","last_undegraded":"2024-03-08T08:28:30.866552+0000","last_fullsized":"2024-03-08T08:28:30.866552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884768+0000","last_change":"2024-03-08T08:28:30.879905+0000","last_active":"2024-03-08T08:28:30.884768+0000","last_peered":"2024-03-08T08:28:30.884768+0000","last_clean":"2024-03-08T08:28:30.884768+0000","last_became_active":"2024-03-08T08:28:30.879646+0000","last_became_peered":"2024-03-08T08:28:30.879646+0000","last_unstale":"2024-03-08T08:28:30.884768+0000","last_undegraded":"2024-03-08T08:28:30.884768+0000","last_fullsized":"2024-03-08T08:28:30.884768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863054+0000","last_change":"2024-03-08T08:28:30.863054+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863054+0000","last_undegraded":"2024-03-08T08:28:30.863054+0000","last_fullsized":"2024-03-08T08:28:30.863054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874360+0000","last_change":"2024-03-08T08:28:30.869336+0000","last_active":"2024-03-08T08:28:30.874360+0000","last_peered":"2024-03-08T08:28:30.874360+0000","last_clean":"2024-03-08T08:28:30.874360+0000","last_became_active":"2024-03-08T08:28:30.868819+0000","last_became_peered":"2024-03-08T08:28:30.868819+0000","last_unstale":"2024-03-08T08:28:30.874360+0000","last_undegraded":"2024-03-08T08:28:30.874360+0000","last_fullsized":"2024-03-08T08:28:30.874360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:04:25.836412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867856+0000","last_change":"2024-03-08T08:28:30.867856+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867856+0000","last_undegraded":"2024-03-08T08:28:30.867856+0000","last_fullsized":"2024-03-08T08:28:30.867856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.872775+0000","last_change":"2024-03-08T08:28:30.866735+0000","last_active":"2024-03-08T08:28:30.872775+0000","last_peered":"2024-03-08T08:28:30.872775+0000","last_clean":"2024-03-08T08:28:30.872775+0000","last_became_active":"2024-03-08T08:28:30.866479+0000","last_became_peered":"2024-03-08T08:28:30.866479+0000","last_unstale":"2024-03-08T08:28:30.872775+0000","last_undegraded":"2024-03-08T08:28:30.872775+0000","last_fullsized":"2024-03-08T08:28:30.872775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:55.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.873789+0000","last_change":"2024-03-08T08:28:30.873789+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.873789+0000","last_undegraded":"2024-03-08T08:28:30.873789+0000","last_fullsized":"2024-03-08T08:28:30.873789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884793+0000","last_change":"2024-03-08T08:28:30.881378+0000","last_active":"2024-03-08T08:28:30.884793+0000","last_peered":"2024-03-08T08:28:30.884793+0000","last_clean":"2024-03-08T08:28:30.884793+0000","last_became_active":"2024-03-08T08:28:30.880959+0000","last_became_peered":"2024-03-08T08:28:30.880959+0000","last_unstale":"2024-03-08T08:28:30.884793+0000","last_undegraded":"2024-03-08T08:28:30.884793+0000","last_fullsized":"2024-03-08T08:28:30.884793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:06:02.644309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.859877+0000","last_change":"2024-03-08T08:28:30.859877+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.859877+0000","last_undegraded":"2024-03-08T08:28:30.859877+0000","last_fullsized":"2024-03-08T08:28:30.859877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.878480+0000","last_change":"2024-03-08T08:28:30.875030+0000","last_active":"2024-03-08T08:28:30.878480+0000","last_peered":"2024-03-08T08:28:30.878480+0000","last_clean":"2024-03-08T08:28:30.878480+0000","last_became_active":"2024-03-08T08:28:30.874753+0000","last_became_peered":"2024-03-08T08:28:30.874753+0000","last_unstale":"2024-03-08T08:28:30.878480+0000","last_undegraded":"2024-03-08T08:28:30.878480+0000","last_fullsized":"2024-03-08T08:28:30.878480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862859+0000","last_change":"2024-03-08T08:28:30.862859+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862859+0000","last_undegraded":"2024-03-08T08:28:30.862859+0000","last_fullsized":"2024-03-08T08:28:30.862859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879326+0000","last_change":"2024-03-08T08:28:30.873585+0000","last_active":"2024-03-08T08:28:30.879326+0000","last_peered":"2024-03-08T08:28:30.879326+0000","last_clean":"2024-03-08T08:28:30.879326+0000","last_became_active":"2024-03-08T08:28:30.873290+0000","last_became_peered":"2024-03-08T08:28:30.873290+0000","last_unstale":"2024-03-08T08:28:30.879326+0000","last_undegraded":"2024-03-08T08:28:30.879326+0000","last_fullsized":"2024-03-08T08:28:30.879326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:25:07.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866610+0000","last_change":"2024-03-08T08:28:30.866610+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866610+0000","last_undegraded":"2024-03-08T08:28:30.866610+0000","last_fullsized":"2024-03-08T08:28:30.866610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874523+0000","last_change":"2024-03-08T08:28:30.871544+0000","last_active":"2024-03-08T08:28:30.874523+0000","last_peered":"2024-03-08T08:28:30.874523+0000","last_clean":"2024-03-08T08:28:30.874523+0000","last_became_active":"2024-03-08T08:28:30.871236+0000","last_became_peered":"2024-03-08T08:28:30.871236+0000","last_unstale":"2024-03-08T08:28:30.874523+0000","last_undegraded":"2024-03-08T08:28:30.874523+0000","last_fullsized":"2024-03-08T08:28:30.874523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:51.840800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862334+0000","last_change":"2024-03-08T08:28:30.862334+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862334+0000","last_undegraded":"2024-03-08T08:28:30.862334+0000","last_fullsized":"2024-03-08T08:28:30.862334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874432+0000","last_change":"2024-03-08T08:28:30.869691+0000","last_active":"2024-03-08T08:28:30.874432+0000","last_peered":"2024-03-08T08:28:30.874432+0000","last_clean":"2024-03-08T08:28:30.874432+0000","last_became_active":"2024-03-08T08:28:30.869367+0000","last_became_peered":"2024-03-08T08:28:30.869367+0000","last_unstale":"2024-03-08T08:28:30.874432+0000","last_undegraded":"2024-03-08T08:28:30.874432+0000","last_fullsized":"2024-03-08T08:28:30.874432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:22:49.859306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865868+0000","last_change":"2024-03-08T08:28:30.865868+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865868+0000","last_undegraded":"2024-03-08T08:28:30.865868+0000","last_fullsized":"2024-03-08T08:28:30.865868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879893+0000","last_change":"2024-03-08T08:28:30.874068+0000","last_active":"2024-03-08T08:28:30.879893+0000","last_peered":"2024-03-08T08:28:30.879893+0000","last_clean":"2024-03-08T08:28:30.879893+0000","last_became_active":"2024-03-08T08:28:30.873748+0000","last_became_peered":"2024-03-08T08:28:30.873748+0000","last_unstale":"2024-03-08T08:28:30.879893+0000","last_undegraded":"2024-03-08T08:28:30.879893+0000","last_fullsized":"2024-03-08T08:28:30.879893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:13:34.320684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862899+0000","last_change":"2024-03-08T08:28:30.862899+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862899+0000","last_undegraded":"2024-03-08T08:28:30.862899+0000","last_fullsized":"2024-03-08T08:28:30.862899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882325+0000","last_change":"2024-03-08T08:28:30.880129+0000","last_active":"2024-03-08T08:28:30.882325+0000","last_peered":"2024-03-08T08:28:30.882325+0000","last_clean":"2024-03-08T08:28:30.882325+0000","last_became_active":"2024-03-08T08:28:30.879847+0000","last_became_peered":"2024-03-08T08:28:30.879847+0000","last_unstale":"2024-03-08T08:28:30.882325+0000","last_undegraded":"2024-03-08T08:28:30.882325+0000","last_fullsized":"2024-03-08T08:28:30.882325+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862815+0000","last_change":"2024-03-08T08:28:30.862815+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862815+0000","last_undegraded":"2024-03-08T08:28:30.862815+0000","last_fullsized":"2024-03-08T08:28:30.862815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.868560+0000","last_change":"2024-03-08T08:28:30.866833+0000","last_active":"2024-03-08T08:28:30.868560+0000","last_peered":"2024-03-08T08:28:30.868560+0000","last_clean":"2024-03-08T08:28:30.868560+0000","last_became_active":"2024-03-08T08:28:30.866548+0000","last_became_peered":"2024-03-08T08:28:30.866548+0000","last_unstale":"2024-03-08T08:28:30.868560+0000","last_undegraded":"2024-03-08T08:28:30.868560+0000","last_fullsized":"2024-03-08T08:28:30.868560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863027+0000","last_change":"2024-03-08T08:28:30.863027+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863027+0000","last_undegraded":"2024-03-08T08:28:30.863027+0000","last_fullsized":"2024-03-08T08:28:30.863027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874381+0000","last_change":"2024-03-08T08:28:30.869452+0000","last_active":"2024-03-08T08:28:30.874381+0000","last_peered":"2024-03-08T08:28:30.874381+0000","last_clean":"2024-03-08T08:28:30.874381+0000","last_became_active":"2024-03-08T08:28:30.868730+0000","last_became_peered":"2024-03-08T08:28:30.868730+0000","last_unstale":"2024-03-08T08:28:30.874381+0000","last_undegraded":"2024-03-08T08:28:30.874381+0000","last_fullsized":"2024-03-08T08:28:30.874381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:37:44.211703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.869141+0000","last_change":"2024-03-08T08:28:30.869141+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.869141+0000","last_undegraded":"2024-03-08T08:28:30.869141+0000","last_fullsized":"2024-03-08T08:28:30.869141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.887238+0000","last_change":"2024-03-08T08:28:30.881897+0000","last_active":"2024-03-08T08:28:30.887238+0000","last_peered":"2024-03-08T08:28:30.887238+0000","last_clean":"2024-03-08T08:28:30.887238+0000","last_became_active":"2024-03-08T08:28:30.881553+0000","last_became_peered":"2024-03-08T08:28:30.881553+0000","last_unstale":"2024-03-08T08:28:30.887238+0000","last_undegraded":"2024-03-08T08:28:30.887238+0000","last_fullsized":"2024-03-08T08:28:30.887238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864018+0000","last_change":"2024-03-08T08:28:30.864018+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864018+0000","last_undegraded":"2024-03-08T08:28:30.864018+0000","last_fullsized":"2024-03-08T08:28:30.864018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.876217+0000","last_change":"2024-03-08T08:28:30.869401+0000","last_active":"2024-03-08T08:28:30.876217+0000","last_peered":"2024-03-08T08:28:30.876217+0000","last_clean":"2024-03-08T08:28:30.876217+0000","last_became_active":"2024-03-08T08:28:30.869130+0000","last_became_peered":"2024-03-08T08:28:30.869130+0000","last_unstale":"2024-03-08T08:28:30.876217+0000","last_undegraded":"2024-03-08T08:28:30.876217+0000","last_fullsized":"2024-03-08T08:28:30.876217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866183+0000","last_change":"2024-03-08T08:28:30.866183+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866183+0000","last_undegraded":"2024-03-08T08:28:30.866183+0000","last_fullsized":"2024-03-08T08:28:30.866183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882345+0000","last_change":"2024-03-08T08:28:30.880219+0000","last_active":"2024-03-08T08:28:30.882345+0000","last_peered":"2024-03-08T08:28:30.882345+0000","last_clean":"2024-03-08T08:28:30.882345+0000","last_became_active":"2024-03-08T08:28:30.879954+0000","last_became_peered":"2024-03-08T08:28:30.879954+0000","last_unstale":"2024-03-08T08:28:30.882345+0000","last_undegraded":"2024-03-08T08:28:30.882345+0000","last_fullsized":"2024-03-08T08:28:30.882345+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864671+0000","last_change":"2024-03-08T08:28:30.864671+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864671+0000","last_undegraded":"2024-03-08T08:28:30.864671+0000","last_fullsized":"2024-03-08T08:28:30.864671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879349+0000","last_change":"2024-03-08T08:28:30.873467+0000","last_active":"2024-03-08T08:28:30.879349+0000","last_peered":"2024-03-08T08:28:30.879349+0000","last_clean":"2024-03-08T08:28:30.879349+0000","last_became_active":"2024-03-08T08:28:30.873184+0000","last_became_peered":"2024-03-08T08:28:30.873184+0000","last_unstale":"2024-03-08T08:28:30.879349+0000","last_undegraded":"2024-03-08T08:28:30.879349+0000","last_fullsized":"2024-03-08T08:28:30.879349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:05:56.384648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860898+0000","last_change":"2024-03-08T08:28:30.860898+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860898+0000","last_undegraded":"2024-03-08T08:28:30.860898+0000","last_fullsized":"2024-03-08T08:28:30.860898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882365+0000","last_change":"2024-03-08T08:28:30.880960+0000","last_active":"2024-03-08T08:28:30.882365+0000","last_peered":"2024-03-08T08:28:30.882365+0000","last_clean":"2024-03-08T08:28:30.882365+0000","last_became_active":"2024-03-08T08:28:30.880615+0000","last_became_peered":"2024-03-08T08:28:30.880615+0000","last_unstale":"2024-03-08T08:28:30.882365+0000","last_undegraded":"2024-03-08T08:28:30.882365+0000","last_fullsized":"2024-03-08T08:28:30.882365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:55.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.871147+0000","last_change":"2024-03-08T08:28:30.871147+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.871147+0000","last_undegraded":"2024-03-08T08:28:30.871147+0000","last_fullsized":"2024-03-08T08:28:30.871147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879388+0000","last_change":"2024-03-08T08:28:30.873910+0000","last_active":"2024-03-08T08:28:30.879388+0000","last_peered":"2024-03-08T08:28:30.879388+0000","last_clean":"2024-03-08T08:28:30.879388+0000","last_became_active":"2024-03-08T08:28:30.872969+0000","last_became_peered":"2024-03-08T08:28:30.872969+0000","last_unstale":"2024-03-08T08:28:30.879388+0000","last_undegraded":"2024-03-08T08:28:30.879388+0000","last_fullsized":"2024-03-08T08:28:30.879388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864159+0000","last_change":"2024-03-08T08:28:30.864159+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864159+0000","last_undegraded":"2024-03-08T08:28:30.864159+0000","last_fullsized":"2024-03-08T08:28:30.864159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874313+0000","last_change":"2024-03-08T08:28:30.869083+0000","last_active":"2024-03-08T08:28:30.874313+0000","last_peered":"2024-03-08T08:28:30.874313+0000","last_clean":"2024-03-08T08:28:30.874313+0000","last_became_active":"2024-03-08T08:28:30.868810+0000","last_became_peered":"2024-03-08T08:28:30.868810+0000","last_unstale":"2024-03-08T08:28:30.874313+0000","last_undegraded":"2024-03-08T08:28:30.874313+0000","last_fullsized":"2024-03-08T08:28:30.874313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:56.443335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862145+0000","last_change":"2024-03-08T08:28:30.862145+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862145+0000","last_undegraded":"2024-03-08T08:28:30.862145+0000","last_fullsized":"2024-03-08T08:28:30.862145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884834+0000","last_change":"2024-03-08T08:28:30.881900+0000","last_active":"2024-03-08T08:28:30.884834+0000","last_peered":"2024-03-08T08:28:30.884834+0000","last_clean":"2024-03-08T08:28:30.884834+0000","last_became_active":"2024-03-08T08:28:30.881645+0000","last_became_peered":"2024-03-08T08:28:30.881645+0000","last_unstale":"2024-03-08T08:28:30.884834+0000","last_undegraded":"2024-03-08T08:28:30.884834+0000","last_fullsized":"2024-03-08T08:28:30.884834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:48:18.245357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868326+0000","last_change":"2024-03-08T08:28:30.868326+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868326+0000","last_undegraded":"2024-03-08T08:28:30.868326+0000","last_fullsized":"2024-03-08T08:28:30.868326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879950+0000","last_change":"2024-03-08T08:28:30.873407+0000","last_active":"2024-03-08T08:28:30.879950+0000","last_peered":"2024-03-08T08:28:30.879950+0000","last_clean":"2024-03-08T08:28:30.879950+0000","last_became_active":"2024-03-08T08:28:30.871776+0000","last_became_peered":"2024-03-08T08:28:30.871776+0000","last_unstale":"2024-03-08T08:28:30.879950+0000","last_undegraded":"2024-03-08T08:28:30.879950+0000","last_fullsized":"2024-03-08T08:28:30.879950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874963+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:28:30.874963+0000","last_peered":"2024-03-08T08:28:30.874963+0000","last_clean":"2024-03-08T08:28:30.874963+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:28:30.874963+0000","last_undegraded":"2024-03-08T08:28:30.874963+0000","last_fullsized":"2024-03-08T08:28:30.874963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:28:35.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-03-08T08:28:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.860746+0000', 'last_change': '2024-03-08T08:28:30.860746+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.860746+0000', 'last_undegraded': '2024-03-08T08:28:30.860746+0000', 'last_fullsized': '2024-03-08T08:28:30.860746+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-08T08:28:35.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.867282+0000', 'last_change': '2024-03-08T08:28:30.867282+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.867282+0000', 'last_undegraded': '2024-03-08T08:28:30.867282+0000', 'last_fullsized': '2024-03-08T08:28:30.867282+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.866105+0000', 'last_change': '2024-03-08T08:28:30.866105+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.866105+0000', 'last_undegraded': '2024-03-08T08:28:30.866105+0000', 'last_fullsized': '2024-03-08T08:28:30.866105+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.867093+0000', 'last_change': '2024-03-08T08:28:30.867093+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.867093+0000', 'last_undegraded': '2024-03-08T08:28:30.867093+0000', 'last_fullsized': '2024-03-08T08:28:30.867093+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.865930+0000', 'last_change': '2024-03-08T08:28:30.865930+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.865930+0000', 'last_undegraded': '2024-03-08T08:28:30.865930+0000', 'last_fullsized': '2024-03-08T08:28:30.865930+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-03-08T08:28:35.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.865937+0000', 'last_change': '2024-03-08T08:28:30.865937+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.865937+0000', 'last_undegraded': '2024-03-08T08:28:30.865937+0000', 'last_fullsized': '2024-03-08T08:28:30.865937+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.868703+0000', 'last_change': '2024-03-08T08:28:30.868703+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.868703+0000', 'last_undegraded': '2024-03-08T08:28:30.868703+0000', 'last_fullsized': '2024-03-08T08:28:30.868703+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.874718+0000', 'last_change': '2024-03-08T08:28:30.874718+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.874718+0000', 'last_undegraded': '2024-03-08T08:28:30.874718+0000', 'last_fullsized': '2024-03-08T08:28:30.874718+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.862649+0000', 'last_change': '2024-03-08T08:28:30.862649+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.862649+0000', 'last_undegraded': '2024-03-08T08:28:30.862649+0000', 'last_fullsized': '2024-03-08T08:28:30.862649+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864662+0000', 'last_change': '2024-03-08T08:28:30.864662+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864662+0000', 'last_undegraded': '2024-03-08T08:28:30.864662+0000', 'last_fullsized': '2024-03-08T08:28:30.864662+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.870446+0000', 'last_change': '2024-03-08T08:28:30.870446+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.870446+0000', 'last_undegraded': '2024-03-08T08:28:30.870446+0000', 'last_fullsized': '2024-03-08T08:28:30.870446+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.865570+0000', 'last_change': '2024-03-08T08:28:30.865570+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.865570+0000', 'last_undegraded': '2024-03-08T08:28:30.865570+0000', 'last_fullsized': '2024-03-08T08:28:30.865570+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.863391+0000', 'last_change': '2024-03-08T08:28:30.863391+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.863391+0000', 'last_undegraded': '2024-03-08T08:28:30.863391+0000', 'last_fullsized': '2024-03-08T08:28:30.863391+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.865483+0000', 'last_change': '2024-03-08T08:28:30.865483+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.865483+0000', 'last_undegraded': '2024-03-08T08:28:30.865483+0000', 'last_fullsized': '2024-03-08T08:28:30.865483+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.865601+0000', 'last_change': '2024-03-08T08:28:30.865601+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.865601+0000', 'last_undegraded': '2024-03-08T08:28:30.865601+0000', 'last_fullsized': '2024-03-08T08:28:30.865601+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.868891+0000', 'last_change': '2024-03-08T08:28:30.868891+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.868891+0000', 'last_undegraded': '2024-03-08T08:28:30.868891+0000', 'last_fullsized': '2024-03-08T08:28:30.868891+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.868737+0000', 'last_change': '2024-03-08T08:28:30.868737+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.868737+0000', 'last_undegraded': '2024-03-08T08:28:30.868737+0000', 'last_fullsized': '2024-03-08T08:28:30.868737+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.866857+0000', 'last_change': '2024-03-08T08:28:30.866857+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.866857+0000', 'last_undegraded': '2024-03-08T08:28:30.866857+0000', 'last_fullsized': '2024-03-08T08:28:30.866857+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.867802+0000', 'last_change': '2024-03-08T08:28:30.867802+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.867802+0000', 'last_undegraded': '2024-03-08T08:28:30.867802+0000', 'last_fullsized': '2024-03-08T08:28:30.867802+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864907+0000', 'last_change': '2024-03-08T08:28:30.864907+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864907+0000', 'last_undegraded': '2024-03-08T08:28:30.864907+0000', 'last_fullsized': '2024-03-08T08:28:30.864907+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864118+0000', 'last_change': '2024-03-08T08:28:30.864118+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864118+0000', 'last_undegraded': '2024-03-08T08:28:30.864118+0000', 'last_fullsized': '2024-03-08T08:28:30.864118+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.862637+0000', 'last_change': '2024-03-08T08:28:30.862637+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.862637+0000', 'last_undegraded': '2024-03-08T08:28:30.862637+0000', 'last_fullsized': '2024-03-08T08:28:30.862637+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.863989+0000', 'last_change': '2024-03-08T08:28:30.863989+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.863989+0000', 'last_undegraded': '2024-03-08T08:28:30.863989+0000', 'last_fullsized': '2024-03-08T08:28:30.863989+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-08T08:28:35.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864765+0000', 'last_change': '2024-03-08T08:28:30.864765+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864765+0000', 'last_undegraded': '2024-03-08T08:28:30.864765+0000', 'last_fullsized': '2024-03-08T08:28:30.864765+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864158+0000', 'last_change': '2024-03-08T08:28:30.864158+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864158+0000', 'last_undegraded': '2024-03-08T08:28:30.864158+0000', 'last_fullsized': '2024-03-08T08:28:30.864158+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.866443+0000', 'last_change': '2024-03-08T08:28:30.866443+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.866443+0000', 'last_undegraded': '2024-03-08T08:28:30.866443+0000', 'last_fullsized': '2024-03-08T08:28:30.866443+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864121+0000', 'last_change': '2024-03-08T08:28:30.864121+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864121+0000', 'last_undegraded': '2024-03-08T08:28:30.864121+0000', 'last_fullsized': '2024-03-08T08:28:30.864121+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864738+0000', 'last_change': '2024-03-08T08:28:30.864738+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864738+0000', 'last_undegraded': '2024-03-08T08:28:30.864738+0000', 'last_fullsized': '2024-03-08T08:28:30.864738+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.863123+0000', 'last_change': '2024-03-08T08:28:30.863123+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.863123+0000', 'last_undegraded': '2024-03-08T08:28:30.863123+0000', 'last_fullsized': '2024-03-08T08:28:30.863123+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-08T08:28:35.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.869203+0000', 'last_change': '2024-03-08T08:28:30.869203+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.869203+0000', 'last_undegraded': '2024-03-08T08:28:30.869203+0000', 'last_fullsized': '2024-03-08T08:28:30.869203+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.860468+0000', 'last_change': '2024-03-08T08:28:30.860468+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.860468+0000', 'last_undegraded': '2024-03-08T08:28:30.860468+0000', 'last_fullsized': '2024-03-08T08:28:30.860468+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.874262+0000', 'last_change': '2024-03-08T08:28:30.874262+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.874262+0000', 'last_undegraded': '2024-03-08T08:28:30.874262+0000', 'last_fullsized': '2024-03-08T08:28:30.874262+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.870812+0000', 'last_change': '2024-03-08T08:28:30.870812+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.870812+0000', 'last_undegraded': '2024-03-08T08:28:30.870812+0000', 'last_fullsized': '2024-03-08T08:28:30.870812+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.860263+0000', 'last_change': '2024-03-08T08:28:30.860263+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.860263+0000', 'last_undegraded': '2024-03-08T08:28:30.860263+0000', 'last_fullsized': '2024-03-08T08:28:30.860263+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-03-08T08:28:35.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.862889+0000', 'last_change': '2024-03-08T08:28:30.862889+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.862889+0000', 'last_undegraded': '2024-03-08T08:28:30.862889+0000', 'last_fullsized': '2024-03-08T08:28:30.862889+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.863108+0000', 'last_change': '2024-03-08T08:28:30.863108+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.863108+0000', 'last_undegraded': '2024-03-08T08:28:30.863108+0000', 'last_fullsized': '2024-03-08T08:28:30.863108+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.868207+0000', 'last_change': '2024-03-08T08:28:30.868207+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.868207+0000', 'last_undegraded': '2024-03-08T08:28:30.868207+0000', 'last_fullsized': '2024-03-08T08:28:30.868207+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864832+0000', 'last_change': '2024-03-08T08:28:30.864832+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864832+0000', 'last_undegraded': '2024-03-08T08:28:30.864832+0000', 'last_fullsized': '2024-03-08T08:28:30.864832+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.860785+0000', 'last_change': '2024-03-08T08:28:30.860785+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.860785+0000', 'last_undegraded': '2024-03-08T08:28:30.860785+0000', 'last_fullsized': '2024-03-08T08:28:30.860785+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.863736+0000', 'last_change': '2024-03-08T08:28:30.863736+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.863736+0000', 'last_undegraded': '2024-03-08T08:28:30.863736+0000', 'last_fullsized': '2024-03-08T08:28:30.863736+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-08T08:28:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864881+0000', 'last_change': '2024-03-08T08:28:30.864881+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864881+0000', 'last_undegraded': '2024-03-08T08:28:30.864881+0000', 'last_fullsized': '2024-03-08T08:28:30.864881+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.866510+0000', 'last_change': '2024-03-08T08:28:30.866510+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.866510+0000', 'last_undegraded': '2024-03-08T08:28:30.866510+0000', 'last_fullsized': '2024-03-08T08:28:30.866510+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.867480+0000', 'last_change': '2024-03-08T08:28:30.867480+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.867480+0000', 'last_undegraded': '2024-03-08T08:28:30.867480+0000', 'last_fullsized': '2024-03-08T08:28:30.867480+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.866552+0000', 'last_change': '2024-03-08T08:28:30.866552+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.866552+0000', 'last_undegraded': '2024-03-08T08:28:30.866552+0000', 'last_fullsized': '2024-03-08T08:28:30.866552+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.863054+0000', 'last_change': '2024-03-08T08:28:30.863054+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.863054+0000', 'last_undegraded': '2024-03-08T08:28:30.863054+0000', 'last_fullsized': '2024-03-08T08:28:30.863054+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-03-08T08:28:35.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.867856+0000', 'last_change': '2024-03-08T08:28:30.867856+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.867856+0000', 'last_undegraded': '2024-03-08T08:28:30.867856+0000', 'last_fullsized': '2024-03-08T08:28:30.867856+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.873789+0000', 'last_change': '2024-03-08T08:28:30.873789+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.873789+0000', 'last_undegraded': '2024-03-08T08:28:30.873789+0000', 'last_fullsized': '2024-03-08T08:28:30.873789+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.859877+0000', 'last_change': '2024-03-08T08:28:30.859877+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.859877+0000', 'last_undegraded': '2024-03-08T08:28:30.859877+0000', 'last_fullsized': '2024-03-08T08:28:30.859877+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.862859+0000', 'last_change': '2024-03-08T08:28:30.862859+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.862859+0000', 'last_undegraded': '2024-03-08T08:28:30.862859+0000', 'last_fullsized': '2024-03-08T08:28:30.862859+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.866610+0000', 'last_change': '2024-03-08T08:28:30.866610+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.866610+0000', 'last_undegraded': '2024-03-08T08:28:30.866610+0000', 'last_fullsized': '2024-03-08T08:28:30.866610+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.862334+0000', 'last_change': '2024-03-08T08:28:30.862334+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.862334+0000', 'last_undegraded': '2024-03-08T08:28:30.862334+0000', 'last_fullsized': '2024-03-08T08:28:30.862334+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-08T08:28:35.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.865868+0000', 'last_change': '2024-03-08T08:28:30.865868+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.865868+0000', 'last_undegraded': '2024-03-08T08:28:30.865868+0000', 'last_fullsized': '2024-03-08T08:28:30.865868+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.862899+0000', 'last_change': '2024-03-08T08:28:30.862899+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.862899+0000', 'last_undegraded': '2024-03-08T08:28:30.862899+0000', 'last_fullsized': '2024-03-08T08:28:30.862899+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.862815+0000', 'last_change': '2024-03-08T08:28:30.862815+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.862815+0000', 'last_undegraded': '2024-03-08T08:28:30.862815+0000', 'last_fullsized': '2024-03-08T08:28:30.862815+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.863027+0000', 'last_change': '2024-03-08T08:28:30.863027+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.863027+0000', 'last_undegraded': '2024-03-08T08:28:30.863027+0000', 'last_fullsized': '2024-03-08T08:28:30.863027+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.869141+0000', 'last_change': '2024-03-08T08:28:30.869141+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.869141+0000', 'last_undegraded': '2024-03-08T08:28:30.869141+0000', 'last_fullsized': '2024-03-08T08:28:30.869141+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-03-08T08:28:35.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864018+0000', 'last_change': '2024-03-08T08:28:30.864018+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864018+0000', 'last_undegraded': '2024-03-08T08:28:30.864018+0000', 'last_fullsized': '2024-03-08T08:28:30.864018+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.866183+0000', 'last_change': '2024-03-08T08:28:30.866183+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.866183+0000', 'last_undegraded': '2024-03-08T08:28:30.866183+0000', 'last_fullsized': '2024-03-08T08:28:30.866183+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864671+0000', 'last_change': '2024-03-08T08:28:30.864671+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864671+0000', 'last_undegraded': '2024-03-08T08:28:30.864671+0000', 'last_fullsized': '2024-03-08T08:28:30.864671+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.860898+0000', 'last_change': '2024-03-08T08:28:30.860898+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.860898+0000', 'last_undegraded': '2024-03-08T08:28:30.860898+0000', 'last_fullsized': '2024-03-08T08:28:30.860898+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.871147+0000', 'last_change': '2024-03-08T08:28:30.871147+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.871147+0000', 'last_undegraded': '2024-03-08T08:28:30.871147+0000', 'last_fullsized': '2024-03-08T08:28:30.871147+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.864159+0000', 'last_change': '2024-03-08T08:28:30.864159+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.864159+0000', 'last_undegraded': '2024-03-08T08:28:30.864159+0000', 'last_fullsized': '2024-03-08T08:28:30.864159+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.862145+0000', 'last_change': '2024-03-08T08:28:30.862145+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.862145+0000', 'last_undegraded': '2024-03-08T08:28:30.862145+0000', 'last_fullsized': '2024-03-08T08:28:30.862145+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-08T08:28:35.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-03-08T08:28:35.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-03-08T08:28:30.868326+0000', 'last_change': '2024-03-08T08:28:30.868326+0000', 'last_active': '2024-03-08T08:28:30.854233+0000', 'last_peered': '2024-03-08T08:28:30.854233+0000', 'last_clean': '2024-03-08T08:28:30.854233+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-08T08:28:30.868326+0000', 'last_undegraded': '2024-03-08T08:28:30.868326+0000', 'last_fullsized': '2024-03-08T08:28:30.868326+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'last_clean_scrub_stamp': '2024-03-08T08:28:30.854233+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-08T08:28:35.528 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-08T08:28:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.648+0000 7f534ba4e640 1 Processor -- start 2024-03-08T08:28:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.648+0000 7f534ba4e640 1 -- start start 2024-03-08T08:28:35.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f534ba4e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f534415be80 0x7f5344066990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f534ba4e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534406bfb0 0x7f53440d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f534ba4e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53440ae740 con 0x7f534406bb30 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f534ba4e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534415c260 con 0x7f534415be80 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f534ba4e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53440afc30 con 0x7f534406bfb0 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f53497ea640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f534415be80 0x7f5344066990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f5348fe9640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f534406bb30 0x7f53440d0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38084/0 (socket says 172.21.15.105:38084) 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f5348fe9640 1 -- 172.21.15.105:0/984205249 learned_addr learned my addr 172.21.15.105:0/984205249 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f53497ea640 1 -- 172.21.15.105:0/984205249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534406bfb0 msgr2=0x7f53440d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f53497ea640 1 --2- 172.21.15.105:0/984205249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534406bfb0 0x7f53440d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f53497ea640 1 -- 172.21.15.105:0/984205249 >> v1:172.21.15.105:6789/0 conn(0x7f534406bb30 legacy=0x7f53440d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f533bfff640 1 -- 172.21.15.105:0/984205249 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673479727 0 0) 0x7f53440ae740 con 0x7f534406bb30 2024-03-08T08:28:35.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.649+0000 7f53497ea640 1 -- 172.21.15.105:0/984205249 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344072630 con 0x7f534415be80 2024-03-08T08:28:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.650+0000 7f53497ea640 1 --2- 172.21.15.105:0/984205249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534415be80 0x7f5344066990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f53440afdb0 tx=0x7f534002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b0ede32ee2414b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.650+0000 7f533bfff640 1 -- 172.21.15.105:0/984205249 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340031770 con 0x7f534415be80 2024-03-08T08:28:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.650+0000 7f533bfff640 1 -- 172.21.15.105:0/984205249 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340037070 con 0x7f534415be80 2024-03-08T08:28:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.650+0000 7f533bfff640 1 -- 172.21.15.105:0/984205249 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340031b30 con 0x7f534415be80 2024-03-08T08:28:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.651+0000 7f534ba4e640 1 -- 172.21.15.105:0/984205249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534415be80 msgr2=0x7f5344066990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.651+0000 7f534ba4e640 1 --2- 172.21.15.105:0/984205249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534415be80 0x7f5344066990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f53440afdb0 tx=0x7f534002b810 comp rx=0 tx=0).stop 2024-03-08T08:28:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.651+0000 7f534ba4e640 1 -- 172.21.15.105:0/984205249 shutdown_connections 2024-03-08T08:28:35.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.651+0000 7f534ba4e640 1 --2- 172.21.15.105:0/984205249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534406bfb0 0x7f53440d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.651+0000 7f534ba4e640 1 --2- 172.21.15.105:0/984205249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534415be80 0x7f5344066990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.651+0000 7f534ba4e640 1 -- 172.21.15.105:0/984205249 >> 172.21.15.105:0/984205249 conn(0x7f53440b50f0 msgr2=0x7f53440b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.651+0000 7f534ba4e640 1 -- 172.21.15.105:0/984205249 shutdown_connections 2024-03-08T08:28:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.651+0000 7f534ba4e640 1 -- 172.21.15.105:0/984205249 wait complete. 2024-03-08T08:28:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.652+0000 7f534ba4e640 1 Processor -- start 2024-03-08T08:28:35.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.652+0000 7f534ba4e640 1 -- start start 2024-03-08T08:28:35.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f534ba4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f534406bb30 0x7f53440d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f534ba4e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534406bfb0 0x7f53440d0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f534ba4e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534415be80 0x7f53441572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f534ba4e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5344072bf0 con 0x7f534406bfb0 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f534ba4e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5344066ed0 con 0x7f534406bb30 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f534ba4e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5344072ed0 con 0x7f534415be80 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5348fe9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534406bfb0 0x7f53440d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5348fe9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534406bfb0 0x7f53440d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36654/0 (socket says 172.21.15.105:36654) 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5348fe9640 1 -- 172.21.15.105:0/4219868427 learned_addr learned my addr 172.21.15.105:0/4219868427 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f53497ea640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f534406bb30 0x7f53440d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5348fe9640 1 -- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534415be80 msgr2=0x7f53441572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5349feb640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534415be80 0x7f53441572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5348fe9640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534415be80 0x7f53441572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5348fe9640 1 -- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f534406bb30 msgr2=0x7f53440d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5348fe9640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f534406bb30 0x7f53440d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f5348fe9640 1 -- 172.21.15.105:0/4219868427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534415e370 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.653+0000 7f53497ea640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f534406bb30 0x7f53440d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.654+0000 7f5348fe9640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534406bfb0 0x7f53440d0690 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f533400c460 tx=0x7f533400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.654+0000 7f5339ffb640 1 -- 172.21.15.105:0/4219868427 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334018070 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.654+0000 7f5339ffb640 1 -- 172.21.15.105:0/4219868427 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5334007500 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.654+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534415d5b0 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.654+0000 7f5339ffb640 1 -- 172.21.15.105:0/4219868427 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334004a20 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.654+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534415db70 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.655+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5344075120 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.655+0000 7f5339ffb640 1 -- 172.21.15.105:0/4219868427 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f533401b070 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.655+0000 7f5339ffb640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f532806b4d0 0x7f532806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.656+0000 7f5339ffb640 1 -- 172.21.15.105:0/4219868427 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53340893f0 con 0x7f534406bfb0 2024-03-08T08:28:35.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.656+0000 7f53497ea640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f532806b4d0 0x7f532806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:35.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.656+0000 7f53497ea640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f532806b4d0 0x7f532806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5340036000 tx=0x7f5340008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:35.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.660+0000 7f5339ffb640 1 -- 172.21.15.105:0/4219868427 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5334058300 con 0x7f534406bfb0 2024-03-08T08:28:35.850 INFO:tasks.ceph.mds.a.smithi105.stderr:2024-03-08T08:28:35.848+0000 7f3e12474640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T08:28:35.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.898+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f534415b300 con 0x7f534406bfb0 2024-03-08T08:28:35.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.898+0000 7f5339ffb640 1 -- 172.21.15.105:0/4219868427 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1581 (secure 0 0 0) 0x7f5334057c00 con 0x7f534406bfb0 2024-03-08T08:28:35.900 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:35.900 INFO:teuthology.orchestra.run.smithi105.stdout:{"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":80,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1709886446,"num_in_osds":8,"osd_in_since":1709886437,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225292288,"bytes_avail":772868820992,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":13,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5100}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-03-08T08:28:27.105491+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-08T08:28:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f532806b4d0 msgr2=0x7f532806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f532806b4d0 0x7f532806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5340036000 tx=0x7f5340008030 comp rx=0 tx=0).stop 2024-03-08T08:28:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534406bfb0 msgr2=0x7f53440d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534406bfb0 0x7f53440d0690 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f533400c460 tx=0x7f533400c920 comp rx=0 tx=0).stop 2024-03-08T08:28:35.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 shutdown_connections 2024-03-08T08:28:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f532806b4d0 0x7f532806d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f534415be80 0x7f53441572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f534406bfb0 0x7f53440d0690 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 --2- 172.21.15.105:0/4219868427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f534406bb30 0x7f53440d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 >> 172.21.15.105:0/4219868427 conn(0x7f53440b50f0 msgr2=0x7f53440ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 shutdown_connections 2024-03-08T08:28:35.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:35.900+0000 7f534ba4e640 1 -- 172.21.15.105:0/4219868427 wait complete. 2024-03-08T08:28:35.913 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-08T08:28:35.913 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:28:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.011+0000 7f3e7ec46640 1 Processor -- start 2024-03-08T08:28:36.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.011+0000 7f3e7ec46640 1 -- start start 2024-03-08T08:28:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.011+0000 7f3e7ec46640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3e7806bcb0 0x7f3e7806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.011+0000 7f3e7ec46640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e780d48a0 con 0x7f3e78161470 2024-03-08T08:28:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.011+0000 7f3e7ec46640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e780d4b80 con 0x7f3e78161850 2024-03-08T08:28:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.011+0000 7f3e7ec46640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7806c170 con 0x7f3e7806bcb0 2024-03-08T08:28:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e7c9e2640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3e78161470 0x7f3e7806b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38092/0 (socket says 172.21.15.105:38092) 2024-03-08T08:28:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e7c9e2640 1 -- 172.21.15.105:0/4135703977 learned_addr learned my addr 172.21.15.105:0/4135703977 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738694542 0 0) 0x7f3e780d48a0 con 0x7f3e78161470 2024-03-08T08:28:36.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e77fff640 1 --2- 172.21.15.105:0/4135703977 >> v2:172.21.15.144:3300/0 conn(0x7f3e7806bcb0 0x7f3e7806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e68003520 con 0x7f3e78161470 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1453393497 0 0) 0x7f3e68003520 con 0x7f3e78161470 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 >> v2:172.21.15.144:3300/0 conn(0x7f3e7806bcb0 msgr2=0x7f3e7806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 --2- 172.21.15.105:0/4135703977 >> v2:172.21.15.144:3300/0 conn(0x7f3e7806bcb0 0x7f3e7806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 >> v1:172.21.15.144:6789/0 conn(0x7f3e78161850 legacy=0x7f3e780b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7806c450 con 0x7f3e78161470 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f3e6c001670 con 0x7f3e78161470 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 >> v1:172.21.15.105:6789/0 conn(0x7f3e78161470 legacy=0x7f3e7806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e680051c0 0x7f3e6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7806c170 con 0x7f3e680039b0 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6800c160 con 0x7f3e68004800 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6800c360 con 0x7f3e680051c0 2024-03-08T08:28:36.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e7c9e2640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e77fff640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.012+0000 7f3e7d1e3640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e680051c0 0x7f3e6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e7c9e2640 1 -- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e680051c0 msgr2=0x7f3e6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e7c9e2640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e680051c0 0x7f3e6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e7c9e2640 1 -- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e68004800 msgr2=0x7f3e68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e7c9e2640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e7c9e2640 1 -- 172.21.15.105:0/4135703977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68003520 con 0x7f3e680039b0 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e7d1e3640 1 -- 172.21.15.105:0/4135703977 reap_dead start 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e77fff640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e68004800 0x7f3e68004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e7c9e2640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c240 tx=0x7f3e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57a00bbe3a83847 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c007d10 con 0x7f3e680039b0 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c030070 con 0x7f3e680039b0 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.013+0000 7f3e777fe640 1 -- 172.21.15.105:0/4135703977 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c02cdc0 con 0x7f3e680039b0 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.014+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4135703977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e680039b0 msgr2=0x7f3e68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:36.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.014+0000 7f3e7ec46640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c240 tx=0x7f3e68005830 comp rx=0 tx=0).stop 2024-03-08T08:28:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.014+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4135703977 shutdown_connections 2024-03-08T08:28:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.014+0000 7f3e7ec46640 1 --2- 172.21.15.105:0/4135703977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e680039b0 0x7f3e68003dd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.014+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4135703977 >> 172.21.15.105:0/4135703977 conn(0x7f3e780d3ce0 msgr2=0x7f3e780d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.014+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4135703977 shutdown_connections 2024-03-08T08:28:36.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.014+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4135703977 wait complete. 2024-03-08T08:28:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.015+0000 7f3e7ec46640 1 Processor -- start 2024-03-08T08:28:36.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.015+0000 7f3e7ec46640 1 -- start start 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7ec46640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e680039b0 0x7f3e780b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7ec46640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e781f7d70 0x7f3e7815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7ec46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e780b19c0 0x7f3e780b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7ec46640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7806d4f0 con 0x7f3e780b19c0 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7ec46640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7806d370 con 0x7f3e680039b0 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7ec46640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7806d670 con 0x7f3e781f7d70 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7d1e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e780b19c0 0x7f3e780b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7d1e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e780b19c0 0x7f3e780b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36684/0 (socket says 172.21.15.105:36684) 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7d1e3640 1 -- 172.21.15.105:0/4257407210 learned_addr learned my addr 172.21.15.105:0/4257407210 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7c9e2640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e680039b0 0x7f3e780b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e77fff640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e781f7d70 0x7f3e7815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7d1e3640 1 -- 172.21.15.105:0/4257407210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e781f7d70 msgr2=0x7f3e7815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7d1e3640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e781f7d70 0x7f3e7815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7d1e3640 1 -- 172.21.15.105:0/4257407210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e680039b0 msgr2=0x7f3e780b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7d1e3640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e680039b0 0x7f3e780b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.016+0000 7f3e7d1e3640 1 -- 172.21.15.105:0/4257407210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7815e8d0 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.017+0000 7f3e7d1e3640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e780b19c0 0x7f3e780b7d20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3e780d1240 tx=0x7f3e7000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.017+0000 7f3e757fa640 1 -- 172.21.15.105:0/4257407210 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70003970 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.017+0000 7f3e757fa640 1 -- 172.21.15.105:0/4257407210 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70018070 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.017+0000 7f3e757fa640 1 -- 172.21.15.105:0/4257407210 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70003d70 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.017+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e780b21d0 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.017+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e781f7570 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.018+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e78075120 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.018+0000 7f3e757fa640 1 -- 172.21.15.105:0/4257407210 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e700050d0 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.019+0000 7f3e757fa640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e5c06b500 0x7f3e5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.019+0000 7f3e757fa640 1 -- 172.21.15.105:0/4257407210 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e70089ca0 con 0x7f3e780b19c0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.019+0000 7f3e7c9e2640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e5c06b500 0x7f3e5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:36.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.019+0000 7f3e7c9e2640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e5c06b500 0x7f3e5c06d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c034000 tx=0x7f3e6c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:36.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.023+0000 7f3e757fa640 1 -- 172.21.15.105:0/4257407210 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e7815e8d0 con 0x7f3e780b19c0 2024-03-08T08:28:36.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.193+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e780b24b0 con 0x7f3e5c06b500 2024-03-08T08:28:36.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.198+0000 7f3e757fa640 1 -- 172.21.15.105:0/4257407210 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246633 (secure 0 0 0) 0x7f3e780b24b0 con 0x7f3e5c06b500 2024-03-08T08:28:36.203 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:28:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e5c06b500 msgr2=0x7f3e5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e5c06b500 0x7f3e5c06d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c034000 tx=0x7f3e6c009030 comp rx=0 tx=0).stop 2024-03-08T08:28:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e780b19c0 msgr2=0x7f3e780b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e780b19c0 0x7f3e780b7d20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3e780d1240 tx=0x7f3e7000ec10 comp rx=0 tx=0).stop 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 shutdown_connections 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3e5c06b500 0x7f3e5c06d9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e780b19c0 0x7f3e780b7d20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e781f7d70 0x7f3e7815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 --2- 172.21.15.105:0/4257407210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e680039b0 0x7f3e780b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 >> 172.21.15.105:0/4257407210 conn(0x7f3e780d3ce0 msgr2=0x7f3e780ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 shutdown_connections 2024-03-08T08:28:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:36.201+0000 7f3e7ec46640 1 -- 172.21.15.105:0/4257407210 wait complete. 2024-03-08T08:28:36.214 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-03-08T08:28:35.106579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220012,"kb_used_data":3832,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754708,"statfs":{"total":773094113280,"available":772868820992,"internally_reserved":0,"allocated":3923968,"data_stored":2223932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.251472"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860746+0000","last_change":"2024-03-08T08:28:30.860746+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860746+0000","last_undegraded":"2024-03-08T08:28:30.860746+0000","last_fullsized":"2024-03-08T08:28:30.860746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867282+0000","last_change":"2024-03-08T08:28:30.867282+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867282+0000","last_undegraded":"2024-03-08T08:28:30.867282+0000","last_fullsized":"2024-03-08T08:28:30.867282+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866105+0000","last_change":"2024-03-08T08:28:30.866105+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866105+0000","last_undegraded":"2024-03-08T08:28:30.866105+0000","last_fullsized":"2024-03-08T08:28:30.866105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867093+0000","last_change":"2024-03-08T08:28:30.867093+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867093+0000","last_undegraded":"2024-03-08T08:28:30.867093+0000","last_fullsized":"2024-03-08T08:28:30.867093+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865930+0000","last_change":"2024-03-08T08:28:30.865930+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865930+0000","last_undegraded":"2024-03-08T08:28:30.865930+0000","last_fullsized":"2024-03-08T08:28:30.865930+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865937+0000","last_change":"2024-03-08T08:28:30.865937+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865937+0000","last_undegraded":"2024-03-08T08:28:30.865937+0000","last_fullsized":"2024-03-08T08:28:30.865937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868703+0000","last_change":"2024-03-08T08:28:30.868703+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868703+0000","last_undegraded":"2024-03-08T08:28:30.868703+0000","last_fullsized":"2024-03-08T08:28:30.868703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.874718+0000","last_change":"2024-03-08T08:28:30.874718+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.874718+0000","last_undegraded":"2024-03-08T08:28:30.874718+0000","last_fullsized":"2024-03-08T08:28:30.874718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862649+0000","last_change":"2024-03-08T08:28:30.862649+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862649+0000","last_undegraded":"2024-03-08T08:28:30.862649+0000","last_fullsized":"2024-03-08T08:28:30.862649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864662+0000","last_change":"2024-03-08T08:28:30.864662+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864662+0000","last_undegraded":"2024-03-08T08:28:30.864662+0000","last_fullsized":"2024-03-08T08:28:30.864662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.870446+0000","last_change":"2024-03-08T08:28:30.870446+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.870446+0000","last_undegraded":"2024-03-08T08:28:30.870446+0000","last_fullsized":"2024-03-08T08:28:30.870446+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865570+0000","last_change":"2024-03-08T08:28:30.865570+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865570+0000","last_undegraded":"2024-03-08T08:28:30.865570+0000","last_fullsized":"2024-03-08T08:28:30.865570+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863391+0000","last_change":"2024-03-08T08:28:30.863391+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863391+0000","last_undegraded":"2024-03-08T08:28:30.863391+0000","last_fullsized":"2024-03-08T08:28:30.863391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865483+0000","last_change":"2024-03-08T08:28:30.865483+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865483+0000","last_undegraded":"2024-03-08T08:28:30.865483+0000","last_fullsized":"2024-03-08T08:28:30.865483+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865601+0000","last_change":"2024-03-08T08:28:30.865601+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865601+0000","last_undegraded":"2024-03-08T08:28:30.865601+0000","last_fullsized":"2024-03-08T08:28:30.865601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868891+0000","last_change":"2024-03-08T08:28:30.868891+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868891+0000","last_undegraded":"2024-03-08T08:28:30.868891+0000","last_fullsized":"2024-03-08T08:28:30.868891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868737+0000","last_change":"2024-03-08T08:28:30.868737+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868737+0000","last_undegraded":"2024-03-08T08:28:30.868737+0000","last_fullsized":"2024-03-08T08:28:30.868737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866857+0000","last_change":"2024-03-08T08:28:30.866857+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866857+0000","last_undegraded":"2024-03-08T08:28:30.866857+0000","last_fullsized":"2024-03-08T08:28:30.866857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867802+0000","last_change":"2024-03-08T08:28:30.867802+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867802+0000","last_undegraded":"2024-03-08T08:28:30.867802+0000","last_fullsized":"2024-03-08T08:28:30.867802+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864907+0000","last_change":"2024-03-08T08:28:30.864907+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864907+0000","last_undegraded":"2024-03-08T08:28:30.864907+0000","last_fullsized":"2024-03-08T08:28:30.864907+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864118+0000","last_change":"2024-03-08T08:28:30.864118+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864118+0000","last_undegraded":"2024-03-08T08:28:30.864118+0000","last_fullsized":"2024-03-08T08:28:30.864118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862637+0000","last_change":"2024-03-08T08:28:30.862637+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862637+0000","last_undegraded":"2024-03-08T08:28:30.862637+0000","last_fullsized":"2024-03-08T08:28:30.862637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863989+0000","last_change":"2024-03-08T08:28:30.863989+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863989+0000","last_undegraded":"2024-03-08T08:28:30.863989+0000","last_fullsized":"2024-03-08T08:28:30.863989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864765+0000","last_change":"2024-03-08T08:28:30.864765+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864765+0000","last_undegraded":"2024-03-08T08:28:30.864765+0000","last_fullsized":"2024-03-08T08:28:30.864765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864158+0000","last_change":"2024-03-08T08:28:30.864158+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864158+0000","last_undegraded":"2024-03-08T08:28:30.864158+0000","last_fullsized":"2024-03-08T08:28:30.864158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866443+0000","last_change":"2024-03-08T08:28:30.866443+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866443+0000","last_undegraded":"2024-03-08T08:28:30.866443+0000","last_fullsized":"2024-03-08T08:28:30.866443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864121+0000","last_change":"2024-03-08T08:28:30.864121+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864121+0000","last_undegraded":"2024-03-08T08:28:30.864121+0000","last_fullsized":"2024-03-08T08:28:30.864121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864738+0000","last_change":"2024-03-08T08:28:30.864738+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864738+0000","last_undegraded":"2024-03-08T08:28:30.864738+0000","last_fullsized":"2024-03-08T08:28:30.864738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863123+0000","last_change":"2024-03-08T08:28:30.863123+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863123+0000","last_undegraded":"2024-03-08T08:28:30.863123+0000","last_fullsized":"2024-03-08T08:28:30.863123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.869203+0000","last_change":"2024-03-08T08:28:30.869203+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.869203+0000","last_undegraded":"2024-03-08T08:28:30.869203+0000","last_fullsized":"2024-03-08T08:28:30.869203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860468+0000","last_change":"2024-03-08T08:28:30.860468+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860468+0000","last_undegraded":"2024-03-08T08:28:30.860468+0000","last_fullsized":"2024-03-08T08:28:30.860468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.874262+0000","last_change":"2024-03-08T08:28:30.874262+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.874262+0000","last_undegraded":"2024-03-08T08:28:30.874262+0000","last_fullsized":"2024-03-08T08:28:30.874262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.870812+0000","last_change":"2024-03-08T08:28:30.870812+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.870812+0000","last_undegraded":"2024-03-08T08:28:30.870812+0000","last_fullsized":"2024-03-08T08:28:30.870812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874668+0000","last_change":"2024-03-08T08:28:30.869323+0000","last_active":"2024-03-08T08:28:30.874668+0000","last_peered":"2024-03-08T08:28:30.874668+0000","last_clean":"2024-03-08T08:28:30.874668+0000","last_became_active":"2024-03-08T08:28:30.869040+0000","last_became_peered":"2024-03-08T08:28:30.869040+0000","last_unstale":"2024-03-08T08:28:30.874668+0000","last_undegraded":"2024-03-08T08:28:30.874668+0000","last_fullsized":"2024-03-08T08:28:30.874668+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860263+0000","last_change":"2024-03-08T08:28:30.860263+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860263+0000","last_undegraded":"2024-03-08T08:28:30.860263+0000","last_fullsized":"2024-03-08T08:28:30.860263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882378+0000","last_change":"2024-03-08T08:28:30.880963+0000","last_active":"2024-03-08T08:28:30.882378+0000","last_peered":"2024-03-08T08:28:30.882378+0000","last_clean":"2024-03-08T08:28:30.882378+0000","last_became_active":"2024-03-08T08:28:30.880613+0000","last_became_peered":"2024-03-08T08:28:30.880613+0000","last_unstale":"2024-03-08T08:28:30.882378+0000","last_undegraded":"2024-03-08T08:28:30.882378+0000","last_fullsized":"2024-03-08T08:28:30.882378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862889+0000","last_change":"2024-03-08T08:28:30.862889+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862889+0000","last_undegraded":"2024-03-08T08:28:30.862889+0000","last_fullsized":"2024-03-08T08:28:30.862889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882179+0000","last_change":"2024-03-08T08:28:30.877050+0000","last_active":"2024-03-08T08:28:30.882179+0000","last_peered":"2024-03-08T08:28:30.882179+0000","last_clean":"2024-03-08T08:28:30.882179+0000","last_became_active":"2024-03-08T08:28:30.876755+0000","last_became_peered":"2024-03-08T08:28:30.876755+0000","last_unstale":"2024-03-08T08:28:30.882179+0000","last_undegraded":"2024-03-08T08:28:30.882179+0000","last_fullsized":"2024-03-08T08:28:30.882179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:02.666304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863108+0000","last_change":"2024-03-08T08:28:30.863108+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863108+0000","last_undegraded":"2024-03-08T08:28:30.863108+0000","last_fullsized":"2024-03-08T08:28:30.863108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.876096+0000","last_change":"2024-03-08T08:28:30.873806+0000","last_active":"2024-03-08T08:28:30.876096+0000","last_peered":"2024-03-08T08:28:30.876096+0000","last_clean":"2024-03-08T08:28:30.876096+0000","last_became_active":"2024-03-08T08:28:30.873361+0000","last_became_peered":"2024-03-08T08:28:30.873361+0000","last_unstale":"2024-03-08T08:28:30.876096+0000","last_undegraded":"2024-03-08T08:28:30.876096+0000","last_fullsized":"2024-03-08T08:28:30.876096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:13:34.320684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868207+0000","last_change":"2024-03-08T08:28:30.868207+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868207+0000","last_undegraded":"2024-03-08T08:28:30.868207+0000","last_fullsized":"2024-03-08T08:28:30.868207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.880408+0000","last_change":"2024-03-08T08:28:30.876030+0000","last_active":"2024-03-08T08:28:30.880408+0000","last_peered":"2024-03-08T08:28:30.880408+0000","last_clean":"2024-03-08T08:28:30.880408+0000","last_became_active":"2024-03-08T08:28:30.875703+0000","last_became_peered":"2024-03-08T08:28:30.875703+0000","last_unstale":"2024-03-08T08:28:30.880408+0000","last_undegraded":"2024-03-08T08:28:30.880408+0000","last_fullsized":"2024-03-08T08:28:30.880408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:56.785346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864832+0000","last_change":"2024-03-08T08:28:30.864832+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864832+0000","last_undegraded":"2024-03-08T08:28:30.864832+0000","last_fullsized":"2024-03-08T08:28:30.864832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874795+0000","last_change":"2024-03-08T08:28:30.873717+0000","last_active":"2024-03-08T08:28:30.874795+0000","last_peered":"2024-03-08T08:28:30.874795+0000","last_clean":"2024-03-08T08:28:30.874795+0000","last_became_active":"2024-03-08T08:28:30.873429+0000","last_became_peered":"2024-03-08T08:28:30.873429+0000","last_unstale":"2024-03-08T08:28:30.874795+0000","last_undegraded":"2024-03-08T08:28:30.874795+0000","last_fullsized":"2024-03-08T08:28:30.874795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:06:58.674770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860785+0000","last_change":"2024-03-08T08:28:30.860785+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860785+0000","last_undegraded":"2024-03-08T08:28:30.860785+0000","last_fullsized":"2024-03-08T08:28:30.860785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882264+0000","last_change":"2024-03-08T08:28:30.877804+0000","last_active":"2024-03-08T08:28:30.882264+0000","last_peered":"2024-03-08T08:28:30.882264+0000","last_clean":"2024-03-08T08:28:30.882264+0000","last_became_active":"2024-03-08T08:28:30.877522+0000","last_became_peered":"2024-03-08T08:28:30.877522+0000","last_unstale":"2024-03-08T08:28:30.882264+0000","last_undegraded":"2024-03-08T08:28:30.882264+0000","last_fullsized":"2024-03-08T08:28:30.882264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:39:07.612726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863736+0000","last_change":"2024-03-08T08:28:30.863736+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863736+0000","last_undegraded":"2024-03-08T08:28:30.863736+0000","last_fullsized":"2024-03-08T08:28:30.863736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874386+0000","last_change":"2024-03-08T08:28:30.869588+0000","last_active":"2024-03-08T08:28:30.874386+0000","last_peered":"2024-03-08T08:28:30.874386+0000","last_clean":"2024-03-08T08:28:30.874386+0000","last_became_active":"2024-03-08T08:28:30.869274+0000","last_became_peered":"2024-03-08T08:28:30.869274+0000","last_unstale":"2024-03-08T08:28:30.874386+0000","last_undegraded":"2024-03-08T08:28:30.874386+0000","last_fullsized":"2024-03-08T08:28:30.874386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:31:59.129966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864881+0000","last_change":"2024-03-08T08:28:30.864881+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864881+0000","last_undegraded":"2024-03-08T08:28:30.864881+0000","last_fullsized":"2024-03-08T08:28:30.864881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884799+0000","last_change":"2024-03-08T08:28:30.881215+0000","last_active":"2024-03-08T08:28:30.884799+0000","last_peered":"2024-03-08T08:28:30.884799+0000","last_clean":"2024-03-08T08:28:30.884799+0000","last_became_active":"2024-03-08T08:28:30.880940+0000","last_became_peered":"2024-03-08T08:28:30.880940+0000","last_unstale":"2024-03-08T08:28:30.884799+0000","last_undegraded":"2024-03-08T08:28:30.884799+0000","last_fullsized":"2024-03-08T08:28:30.884799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866510+0000","last_change":"2024-03-08T08:28:30.866510+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866510+0000","last_undegraded":"2024-03-08T08:28:30.866510+0000","last_fullsized":"2024-03-08T08:28:30.866510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.881459+0000","last_change":"2024-03-08T08:28:30.876384+0000","last_active":"2024-03-08T08:28:30.881459+0000","last_peered":"2024-03-08T08:28:30.881459+0000","last_clean":"2024-03-08T08:28:30.881459+0000","last_became_active":"2024-03-08T08:28:30.876053+0000","last_became_peered":"2024-03-08T08:28:30.876053+0000","last_unstale":"2024-03-08T08:28:30.881459+0000","last_undegraded":"2024-03-08T08:28:30.881459+0000","last_fullsized":"2024-03-08T08:28:30.881459+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867480+0000","last_change":"2024-03-08T08:28:30.867480+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867480+0000","last_undegraded":"2024-03-08T08:28:30.867480+0000","last_fullsized":"2024-03-08T08:28:30.867480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.885692+0000","last_change":"2024-03-08T08:28:30.880920+0000","last_active":"2024-03-08T08:28:30.885692+0000","last_peered":"2024-03-08T08:28:30.885692+0000","last_clean":"2024-03-08T08:28:30.885692+0000","last_became_active":"2024-03-08T08:28:30.880573+0000","last_became_peered":"2024-03-08T08:28:30.880573+0000","last_unstale":"2024-03-08T08:28:30.885692+0000","last_undegraded":"2024-03-08T08:28:30.885692+0000","last_fullsized":"2024-03-08T08:28:30.885692+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866552+0000","last_change":"2024-03-08T08:28:30.866552+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866552+0000","last_undegraded":"2024-03-08T08:28:30.866552+0000","last_fullsized":"2024-03-08T08:28:30.866552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884768+0000","last_change":"2024-03-08T08:28:30.879905+0000","last_active":"2024-03-08T08:28:30.884768+0000","last_peered":"2024-03-08T08:28:30.884768+0000","last_clean":"2024-03-08T08:28:30.884768+0000","last_became_active":"2024-03-08T08:28:30.879646+0000","last_became_peered":"2024-03-08T08:28:30.879646+0000","last_unstale":"2024-03-08T08:28:30.884768+0000","last_undegraded":"2024-03-08T08:28:30.884768+0000","last_fullsized":"2024-03-08T08:28:30.884768+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863054+0000","last_change":"2024-03-08T08:28:30.863054+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863054+0000","last_undegraded":"2024-03-08T08:28:30.863054+0000","last_fullsized":"2024-03-08T08:28:30.863054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874360+0000","last_change":"2024-03-08T08:28:30.869336+0000","last_active":"2024-03-08T08:28:30.874360+0000","last_peered":"2024-03-08T08:28:30.874360+0000","last_clean":"2024-03-08T08:28:30.874360+0000","last_became_active":"2024-03-08T08:28:30.868819+0000","last_became_peered":"2024-03-08T08:28:30.868819+0000","last_unstale":"2024-03-08T08:28:30.874360+0000","last_undegraded":"2024-03-08T08:28:30.874360+0000","last_fullsized":"2024-03-08T08:28:30.874360+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:04:25.836412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.867856+0000","last_change":"2024-03-08T08:28:30.867856+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.867856+0000","last_undegraded":"2024-03-08T08:28:30.867856+0000","last_fullsized":"2024-03-08T08:28:30.867856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.872775+0000","last_change":"2024-03-08T08:28:30.866735+0000","last_active":"2024-03-08T08:28:30.872775+0000","last_peered":"2024-03-08T08:28:30.872775+0000","last_clean":"2024-03-08T08:28:30.872775+0000","last_became_active":"2024-03-08T08:28:30.866479+0000","last_became_peered":"2024-03-08T08:28:30.866479+0000","last_unstale":"2024-03-08T08:28:30.872775+0000","last_undegraded":"2024-03-08T08:28:30.872775+0000","last_fullsized":"2024-03-08T08:28:30.872775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:55.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.873789+0000","last_change":"2024-03-08T08:28:30.873789+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.873789+0000","last_undegraded":"2024-03-08T08:28:30.873789+0000","last_fullsized":"2024-03-08T08:28:30.873789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884793+0000","last_change":"2024-03-08T08:28:30.881378+0000","last_active":"2024-03-08T08:28:30.884793+0000","last_peered":"2024-03-08T08:28:30.884793+0000","last_clean":"2024-03-08T08:28:30.884793+0000","last_became_active":"2024-03-08T08:28:30.880959+0000","last_became_peered":"2024-03-08T08:28:30.880959+0000","last_unstale":"2024-03-08T08:28:30.884793+0000","last_undegraded":"2024-03-08T08:28:30.884793+0000","last_fullsized":"2024-03-08T08:28:30.884793+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:06:02.644309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.859877+0000","last_change":"2024-03-08T08:28:30.859877+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.859877+0000","last_undegraded":"2024-03-08T08:28:30.859877+0000","last_fullsized":"2024-03-08T08:28:30.859877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.878480+0000","last_change":"2024-03-08T08:28:30.875030+0000","last_active":"2024-03-08T08:28:30.878480+0000","last_peered":"2024-03-08T08:28:30.878480+0000","last_clean":"2024-03-08T08:28:30.878480+0000","last_became_active":"2024-03-08T08:28:30.874753+0000","last_became_peered":"2024-03-08T08:28:30.874753+0000","last_unstale":"2024-03-08T08:28:30.878480+0000","last_undegraded":"2024-03-08T08:28:30.878480+0000","last_fullsized":"2024-03-08T08:28:30.878480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862859+0000","last_change":"2024-03-08T08:28:30.862859+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862859+0000","last_undegraded":"2024-03-08T08:28:30.862859+0000","last_fullsized":"2024-03-08T08:28:30.862859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879326+0000","last_change":"2024-03-08T08:28:30.873585+0000","last_active":"2024-03-08T08:28:30.879326+0000","last_peered":"2024-03-08T08:28:30.879326+0000","last_clean":"2024-03-08T08:28:30.879326+0000","last_became_active":"2024-03-08T08:28:30.873290+0000","last_became_peered":"2024-03-08T08:28:30.873290+0000","last_unstale":"2024-03-08T08:28:30.879326+0000","last_undegraded":"2024-03-08T08:28:30.879326+0000","last_fullsized":"2024-03-08T08:28:30.879326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:25:07.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866610+0000","last_change":"2024-03-08T08:28:30.866610+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866610+0000","last_undegraded":"2024-03-08T08:28:30.866610+0000","last_fullsized":"2024-03-08T08:28:30.866610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874523+0000","last_change":"2024-03-08T08:28:30.871544+0000","last_active":"2024-03-08T08:28:30.874523+0000","last_peered":"2024-03-08T08:28:30.874523+0000","last_clean":"2024-03-08T08:28:30.874523+0000","last_became_active":"2024-03-08T08:28:30.871236+0000","last_became_peered":"2024-03-08T08:28:30.871236+0000","last_unstale":"2024-03-08T08:28:30.874523+0000","last_undegraded":"2024-03-08T08:28:30.874523+0000","last_fullsized":"2024-03-08T08:28:30.874523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:04:51.840800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862334+0000","last_change":"2024-03-08T08:28:30.862334+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862334+0000","last_undegraded":"2024-03-08T08:28:30.862334+0000","last_fullsized":"2024-03-08T08:28:30.862334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874432+0000","last_change":"2024-03-08T08:28:30.869691+0000","last_active":"2024-03-08T08:28:30.874432+0000","last_peered":"2024-03-08T08:28:30.874432+0000","last_clean":"2024-03-08T08:28:30.874432+0000","last_became_active":"2024-03-08T08:28:30.869367+0000","last_became_peered":"2024-03-08T08:28:30.869367+0000","last_unstale":"2024-03-08T08:28:30.874432+0000","last_undegraded":"2024-03-08T08:28:30.874432+0000","last_fullsized":"2024-03-08T08:28:30.874432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:22:49.859306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.865868+0000","last_change":"2024-03-08T08:28:30.865868+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.865868+0000","last_undegraded":"2024-03-08T08:28:30.865868+0000","last_fullsized":"2024-03-08T08:28:30.865868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879893+0000","last_change":"2024-03-08T08:28:30.874068+0000","last_active":"2024-03-08T08:28:30.879893+0000","last_peered":"2024-03-08T08:28:30.879893+0000","last_clean":"2024-03-08T08:28:30.879893+0000","last_became_active":"2024-03-08T08:28:30.873748+0000","last_became_peered":"2024-03-08T08:28:30.873748+0000","last_unstale":"2024-03-08T08:28:30.879893+0000","last_undegraded":"2024-03-08T08:28:30.879893+0000","last_fullsized":"2024-03-08T08:28:30.879893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:13:34.320684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862899+0000","last_change":"2024-03-08T08:28:30.862899+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862899+0000","last_undegraded":"2024-03-08T08:28:30.862899+0000","last_fullsized":"2024-03-08T08:28:30.862899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882325+0000","last_change":"2024-03-08T08:28:30.880129+0000","last_active":"2024-03-08T08:28:30.882325+0000","last_peered":"2024-03-08T08:28:30.882325+0000","last_clean":"2024-03-08T08:28:30.882325+0000","last_became_active":"2024-03-08T08:28:30.879847+0000","last_became_peered":"2024-03-08T08:28:30.879847+0000","last_unstale":"2024-03-08T08:28:30.882325+0000","last_undegraded":"2024-03-08T08:28:30.882325+0000","last_fullsized":"2024-03-08T08:28:30.882325+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862815+0000","last_change":"2024-03-08T08:28:30.862815+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862815+0000","last_undegraded":"2024-03-08T08:28:30.862815+0000","last_fullsized":"2024-03-08T08:28:30.862815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.868560+0000","last_change":"2024-03-08T08:28:30.866833+0000","last_active":"2024-03-08T08:28:30.868560+0000","last_peered":"2024-03-08T08:28:30.868560+0000","last_clean":"2024-03-08T08:28:30.868560+0000","last_became_active":"2024-03-08T08:28:30.866548+0000","last_became_peered":"2024-03-08T08:28:30.866548+0000","last_unstale":"2024-03-08T08:28:30.868560+0000","last_undegraded":"2024-03-08T08:28:30.868560+0000","last_fullsized":"2024-03-08T08:28:30.868560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.863027+0000","last_change":"2024-03-08T08:28:30.863027+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.863027+0000","last_undegraded":"2024-03-08T08:28:30.863027+0000","last_fullsized":"2024-03-08T08:28:30.863027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874381+0000","last_change":"2024-03-08T08:28:30.869452+0000","last_active":"2024-03-08T08:28:30.874381+0000","last_peered":"2024-03-08T08:28:30.874381+0000","last_clean":"2024-03-08T08:28:30.874381+0000","last_became_active":"2024-03-08T08:28:30.868730+0000","last_became_peered":"2024-03-08T08:28:30.868730+0000","last_unstale":"2024-03-08T08:28:30.874381+0000","last_undegraded":"2024-03-08T08:28:30.874381+0000","last_fullsized":"2024-03-08T08:28:30.874381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:37:44.211703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.869141+0000","last_change":"2024-03-08T08:28:30.869141+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.869141+0000","last_undegraded":"2024-03-08T08:28:30.869141+0000","last_fullsized":"2024-03-08T08:28:30.869141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.887238+0000","last_change":"2024-03-08T08:28:30.881897+0000","last_active":"2024-03-08T08:28:30.887238+0000","last_peered":"2024-03-08T08:28:30.887238+0000","last_clean":"2024-03-08T08:28:30.887238+0000","last_became_active":"2024-03-08T08:28:30.881553+0000","last_became_peered":"2024-03-08T08:28:30.881553+0000","last_unstale":"2024-03-08T08:28:30.887238+0000","last_undegraded":"2024-03-08T08:28:30.887238+0000","last_fullsized":"2024-03-08T08:28:30.887238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864018+0000","last_change":"2024-03-08T08:28:30.864018+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864018+0000","last_undegraded":"2024-03-08T08:28:30.864018+0000","last_fullsized":"2024-03-08T08:28:30.864018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.876217+0000","last_change":"2024-03-08T08:28:30.869401+0000","last_active":"2024-03-08T08:28:30.876217+0000","last_peered":"2024-03-08T08:28:30.876217+0000","last_clean":"2024-03-08T08:28:30.876217+0000","last_became_active":"2024-03-08T08:28:30.869130+0000","last_became_peered":"2024-03-08T08:28:30.869130+0000","last_unstale":"2024-03-08T08:28:30.876217+0000","last_undegraded":"2024-03-08T08:28:30.876217+0000","last_fullsized":"2024-03-08T08:28:30.876217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.866183+0000","last_change":"2024-03-08T08:28:30.866183+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.866183+0000","last_undegraded":"2024-03-08T08:28:30.866183+0000","last_fullsized":"2024-03-08T08:28:30.866183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882345+0000","last_change":"2024-03-08T08:28:30.880219+0000","last_active":"2024-03-08T08:28:30.882345+0000","last_peered":"2024-03-08T08:28:30.882345+0000","last_clean":"2024-03-08T08:28:30.882345+0000","last_became_active":"2024-03-08T08:28:30.879954+0000","last_became_peered":"2024-03-08T08:28:30.879954+0000","last_unstale":"2024-03-08T08:28:30.882345+0000","last_undegraded":"2024-03-08T08:28:30.882345+0000","last_fullsized":"2024-03-08T08:28:30.882345+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864671+0000","last_change":"2024-03-08T08:28:30.864671+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864671+0000","last_undegraded":"2024-03-08T08:28:30.864671+0000","last_fullsized":"2024-03-08T08:28:30.864671+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879349+0000","last_change":"2024-03-08T08:28:30.873467+0000","last_active":"2024-03-08T08:28:30.879349+0000","last_peered":"2024-03-08T08:28:30.879349+0000","last_clean":"2024-03-08T08:28:30.879349+0000","last_became_active":"2024-03-08T08:28:30.873184+0000","last_became_peered":"2024-03-08T08:28:30.873184+0000","last_unstale":"2024-03-08T08:28:30.879349+0000","last_undegraded":"2024-03-08T08:28:30.879349+0000","last_fullsized":"2024-03-08T08:28:30.879349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:05:56.384648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.860898+0000","last_change":"2024-03-08T08:28:30.860898+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.860898+0000","last_undegraded":"2024-03-08T08:28:30.860898+0000","last_fullsized":"2024-03-08T08:28:30.860898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.882365+0000","last_change":"2024-03-08T08:28:30.880960+0000","last_active":"2024-03-08T08:28:30.882365+0000","last_peered":"2024-03-08T08:28:30.882365+0000","last_clean":"2024-03-08T08:28:30.882365+0000","last_became_active":"2024-03-08T08:28:30.880615+0000","last_became_peered":"2024-03-08T08:28:30.880615+0000","last_unstale":"2024-03-08T08:28:30.882365+0000","last_undegraded":"2024-03-08T08:28:30.882365+0000","last_fullsized":"2024-03-08T08:28:30.882365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:55.063584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.871147+0000","last_change":"2024-03-08T08:28:30.871147+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.871147+0000","last_undegraded":"2024-03-08T08:28:30.871147+0000","last_fullsized":"2024-03-08T08:28:30.871147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879388+0000","last_change":"2024-03-08T08:28:30.873910+0000","last_active":"2024-03-08T08:28:30.879388+0000","last_peered":"2024-03-08T08:28:30.879388+0000","last_clean":"2024-03-08T08:28:30.879388+0000","last_became_active":"2024-03-08T08:28:30.872969+0000","last_became_peered":"2024-03-08T08:28:30.872969+0000","last_unstale":"2024-03-08T08:28:30.879388+0000","last_undegraded":"2024-03-08T08:28:30.879388+0000","last_fullsized":"2024-03-08T08:28:30.879388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.864159+0000","last_change":"2024-03-08T08:28:30.864159+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.864159+0000","last_undegraded":"2024-03-08T08:28:30.864159+0000","last_fullsized":"2024-03-08T08:28:30.864159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874313+0000","last_change":"2024-03-08T08:28:30.869083+0000","last_active":"2024-03-08T08:28:30.874313+0000","last_peered":"2024-03-08T08:28:30.874313+0000","last_clean":"2024-03-08T08:28:30.874313+0000","last_became_active":"2024-03-08T08:28:30.868810+0000","last_became_peered":"2024-03-08T08:28:30.868810+0000","last_unstale":"2024-03-08T08:28:30.874313+0000","last_undegraded":"2024-03-08T08:28:30.874313+0000","last_fullsized":"2024-03-08T08:28:30.874313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:23:56.443335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.862145+0000","last_change":"2024-03-08T08:28:30.862145+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.862145+0000","last_undegraded":"2024-03-08T08:28:30.862145+0000","last_fullsized":"2024-03-08T08:28:30.862145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.884834+0000","last_change":"2024-03-08T08:28:30.881900+0000","last_active":"2024-03-08T08:28:30.884834+0000","last_peered":"2024-03-08T08:28:30.884834+0000","last_clean":"2024-03-08T08:28:30.884834+0000","last_became_active":"2024-03-08T08:28:30.881645+0000","last_became_peered":"2024-03-08T08:28:30.881645+0000","last_unstale":"2024-03-08T08:28:30.884834+0000","last_undegraded":"2024-03-08T08:28:30.884834+0000","last_fullsized":"2024-03-08T08:28:30.884834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:48:18.245357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-03-08T08:28:30.868326+0000","last_change":"2024-03-08T08:28:30.868326+0000","last_active":"2024-03-08T08:28:30.854233+0000","last_peered":"2024-03-08T08:28:30.854233+0000","last_clean":"2024-03-08T08:28:30.854233+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-08T08:28:30.868326+0000","last_undegraded":"2024-03-08T08:28:30.868326+0000","last_fullsized":"2024-03-08T08:28:30.868326+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.879950+0000","last_change":"2024-03-08T08:28:30.873407+0000","last_active":"2024-03-08T08:28:30.879950+0000","last_peered":"2024-03-08T08:28:30.879950+0000","last_clean":"2024-03-08T08:28:30.879950+0000","last_became_active":"2024-03-08T08:28:30.871776+0000","last_became_peered":"2024-03-08T08:28:30.871776+0000","last_unstale":"2024-03-08T08:28:30.879950+0000","last_undegraded":"2024-03-08T08:28:30.879950+0000","last_fullsized":"2024-03-08T08:28:30.879950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-08T08:28:30.874963+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:28:30.874963+0000","last_peered":"2024-03-08T08:28:30.874963+0000","last_clean":"2024-03-08T08:28:30.874963+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:28:30.874963+0000","last_undegraded":"2024-03-08T08:28:30.874963+0000","last_fullsized":"2024-03-08T08:28:30.874963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:28:38.912 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:28:38.915 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:28:39.219 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:28:39.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.322+0000 7fb37580f640 1 Processor -- start 2024-03-08T08:28:39.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.322+0000 7fb37580f640 1 -- start start 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb37580f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006fd60 0x7fb3700ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb37580f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb37006ac80 0x7fb37006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb37580f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37006deb0 con 0x7fb3701469e0 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb37580f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3700701c0 con 0x7fb37006ac80 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb37580f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37006e190 con 0x7fb37006fd60 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb36f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb37006ac80 0x7fb37006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb36f7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb37006ac80 0x7fb37006b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36692/0 (socket says 172.21.15.105:36692) 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb36f7fe640 1 -- 172.21.15.105:0/3342744931 learned_addr learned my addr 172.21.15.105:0/3342744931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:39.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb36e7fc640 1 --2- 172.21.15.105:0/3342744931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006fd60 0x7fb3700ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb36f7fe640 1 -- 172.21.15.105:0/3342744931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006fd60 msgr2=0x7fb3700ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:39.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb36f7fe640 1 --2- 172.21.15.105:0/3342744931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006fd60 0x7fb3700ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb36f7fe640 1 -- 172.21.15.105:0/3342744931 >> v1:172.21.15.144:6789/0 conn(0x7fb3701469e0 legacy=0x7fb37006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.323+0000 7fb36f7fe640 1 -- 172.21.15.105:0/3342744931 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700b42f0 con 0x7fb37006ac80 2024-03-08T08:28:39.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.324+0000 7fb36f7fe640 1 --2- 172.21.15.105:0/3342744931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006ac80 0x7fb37006b110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb368009870 tx=0x7fb36802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8747d318dbfe2ea server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:39.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.324+0000 7fb36dffb640 1 -- 172.21.15.105:0/3342744931 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368035020 con 0x7fb37006ac80 2024-03-08T08:28:39.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.324+0000 7fb36dffb640 1 -- 172.21.15.105:0/3342744931 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb368031640 con 0x7fb37006ac80 2024-03-08T08:28:39.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.324+0000 7fb36dffb640 1 -- 172.21.15.105:0/3342744931 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36802d040 con 0x7fb37006ac80 2024-03-08T08:28:39.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.325+0000 7fb37580f640 1 -- 172.21.15.105:0/3342744931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006ac80 msgr2=0x7fb37006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.325+0000 7fb37580f640 1 --2- 172.21.15.105:0/3342744931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006ac80 0x7fb37006b110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb368009870 tx=0x7fb36802b8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.325+0000 7fb37580f640 1 -- 172.21.15.105:0/3342744931 shutdown_connections 2024-03-08T08:28:39.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.325+0000 7fb37580f640 1 --2- 172.21.15.105:0/3342744931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006ac80 0x7fb37006b110 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.325+0000 7fb37580f640 1 --2- 172.21.15.105:0/3342744931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006fd60 0x7fb3700ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.325+0000 7fb37580f640 1 -- 172.21.15.105:0/3342744931 >> 172.21.15.105:0/3342744931 conn(0x7fb3700b2670 msgr2=0x7fb3700b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:39.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.325+0000 7fb37580f640 1 -- 172.21.15.105:0/3342744931 shutdown_connections 2024-03-08T08:28:39.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.325+0000 7fb37580f640 1 -- 172.21.15.105:0/3342744931 wait complete. 2024-03-08T08:28:39.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.326+0000 7fb37580f640 1 Processor -- start 2024-03-08T08:28:39.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.326+0000 7fb37580f640 1 -- start start 2024-03-08T08:28:39.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb37580f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006ac80 0x7fb3701f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb37580f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006fd60 0x7fb3701f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb37580f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3701469e0 0x7fb3700ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb37580f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb370146dc0 con 0x7fb37006fd60 2024-03-08T08:28:39.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb37580f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37006e470 con 0x7fb3701469e0 2024-03-08T08:28:39.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb37580f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700b4af0 con 0x7fb37006ac80 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006fd60 0x7fb3701f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006fd60 0x7fb3701f40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36704/0 (socket says 172.21.15.105:36704) 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 -- 172.21.15.105:0/1144676559 learned_addr learned my addr 172.21.15.105:0/1144676559 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36f7fe640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3701469e0 0x7fb3700ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36effd640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006ac80 0x7fb3701f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 -- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006ac80 msgr2=0x7fb3701f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006ac80 0x7fb3701f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 -- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3701469e0 msgr2=0x7fb3700ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3701469e0 0x7fb3700ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 -- 172.21.15.105:0/1144676559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700cd530 con 0x7fb37006fd60 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36effd640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006ac80 0x7fb3701f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36f7fe640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3701469e0 0x7fb3700ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb36e7fc640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006fd60 0x7fb3701f40a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb36400d800 tx=0x7fb36400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb357fff640 1 -- 172.21.15.105:0/1144676559 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400b6c0 con 0x7fb37006fd60 2024-03-08T08:28:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb357fff640 1 -- 172.21.15.105:0/1144676559 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb364019070 con 0x7fb37006fd60 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3700cd7d0 con 0x7fb37006fd60 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb357fff640 1 -- 172.21.15.105:0/1144676559 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400bac0 con 0x7fb37006fd60 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.327+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3700cdd90 con 0x7fb37006fd60 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.329+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb370075120 con 0x7fb37006fd60 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.329+0000 7fb357fff640 1 -- 172.21.15.105:0/1144676559 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb36400bc60 con 0x7fb37006fd60 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.329+0000 7fb357fff640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35c06b4d0 0x7fb35c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.329+0000 7fb357fff640 1 -- 172.21.15.105:0/1144676559 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb36408a6c0 con 0x7fb37006fd60 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.329+0000 7fb36effd640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35c06b4d0 0x7fb35c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.330+0000 7fb36effd640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35c06b4d0 0x7fb35c06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb360001d60 tx=0x7fb360009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:39.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.334+0000 7fb357fff640 1 -- 172.21.15.105:0/1144676559 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb364059540 con 0x7fb37006fd60 2024-03-08T08:28:39.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.504+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3700b5bb0 con 0x7fb35c06b4d0 2024-03-08T08:28:39.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.509+0000 7fb357fff640 1 -- 172.21.15.105:0/1144676559 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7fb3700b5bb0 con 0x7fb35c06b4d0 2024-03-08T08:28:39.514 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:39.515 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35c06b4d0 msgr2=0x7fb35c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35c06b4d0 0x7fb35c06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb360001d60 tx=0x7fb360009040 comp rx=0 tx=0).stop 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006fd60 msgr2=0x7fb3701f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006fd60 0x7fb3701f40a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb36400d800 tx=0x7fb36400dcc0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 shutdown_connections 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb35c06b4d0 0x7fb35c06d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3701469e0 0x7fb3700ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb37006fd60 0x7fb3701f40a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 --2- 172.21.15.105:0/1144676559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb37006ac80 0x7fb3701f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 >> 172.21.15.105:0/1144676559 conn(0x7fb3700b2670 msgr2=0x7fb370143b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 shutdown_connections 2024-03-08T08:28:39.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.513+0000 7fb37580f640 1 -- 172.21.15.105:0/1144676559 wait complete. 2024-03-08T08:28:39.525 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-03-08T08:28:39.107675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220320,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754400,"statfs":{"total":773094113280,"available":772868505600,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001690"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883977+0000","last_change":"2024-03-08T08:28:31.884197+0000","last_active":"2024-03-08T08:28:32.883977+0000","last_peered":"2024-03-08T08:28:32.883977+0000","last_clean":"2024-03-08T08:28:32.883977+0000","last_became_active":"2024-03-08T08:28:31.883921+0000","last_became_peered":"2024-03-08T08:28:31.883921+0000","last_unstale":"2024-03-08T08:28:32.883977+0000","last_undegraded":"2024-03-08T08:28:32.883977+0000","last_fullsized":"2024-03-08T08:28:32.883977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:10:14.609573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736065+0000","last_change":"2024-03-08T08:28:31.896612+0000","last_active":"2024-03-08T08:28:33.736065+0000","last_peered":"2024-03-08T08:28:33.736065+0000","last_clean":"2024-03-08T08:28:33.736065+0000","last_became_active":"2024-03-08T08:28:31.896330+0000","last_became_peered":"2024-03-08T08:28:31.896330+0000","last_unstale":"2024-03-08T08:28:33.736065+0000","last_undegraded":"2024-03-08T08:28:33.736065+0000","last_fullsized":"2024-03-08T08:28:33.736065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:05:03.670099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736172+0000","last_change":"2024-03-08T08:28:31.890903+0000","last_active":"2024-03-08T08:28:33.736172+0000","last_peered":"2024-03-08T08:28:33.736172+0000","last_clean":"2024-03-08T08:28:33.736172+0000","last_became_active":"2024-03-08T08:28:31.890464+0000","last_became_peered":"2024-03-08T08:28:31.890464+0000","last_unstale":"2024-03-08T08:28:33.736172+0000","last_undegraded":"2024-03-08T08:28:33.736172+0000","last_fullsized":"2024-03-08T08:28:33.736172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:32.298587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883106+0000","last_change":"2024-03-08T08:28:31.893686+0000","last_active":"2024-03-08T08:28:32.883106+0000","last_peered":"2024-03-08T08:28:32.883106+0000","last_clean":"2024-03-08T08:28:32.883106+0000","last_became_active":"2024-03-08T08:28:31.893030+0000","last_became_peered":"2024-03-08T08:28:31.893030+0000","last_unstale":"2024-03-08T08:28:32.883106+0000","last_undegraded":"2024-03-08T08:28:32.883106+0000","last_fullsized":"2024-03-08T08:28:32.883106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:25:43.946833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735415+0000","last_change":"2024-03-08T08:28:31.882439+0000","last_active":"2024-03-08T08:28:33.735415+0000","last_peered":"2024-03-08T08:28:33.735415+0000","last_clean":"2024-03-08T08:28:33.735415+0000","last_became_active":"2024-03-08T08:28:31.882127+0000","last_became_peered":"2024-03-08T08:28:31.882127+0000","last_unstale":"2024-03-08T08:28:33.735415+0000","last_undegraded":"2024-03-08T08:28:33.735415+0000","last_fullsized":"2024-03-08T08:28:33.735415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:39:08.616752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735622+0000","last_change":"2024-03-08T08:28:31.883288+0000","last_active":"2024-03-08T08:28:33.735622+0000","last_peered":"2024-03-08T08:28:33.735622+0000","last_clean":"2024-03-08T08:28:33.735622+0000","last_became_active":"2024-03-08T08:28:31.883042+0000","last_became_peered":"2024-03-08T08:28:31.883042+0000","last_unstale":"2024-03-08T08:28:33.735622+0000","last_undegraded":"2024-03-08T08:28:33.735622+0000","last_fullsized":"2024-03-08T08:28:33.735622+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:32.298587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736003+0000","last_change":"2024-03-08T08:28:31.888167+0000","last_active":"2024-03-08T08:28:33.736003+0000","last_peered":"2024-03-08T08:28:33.736003+0000","last_clean":"2024-03-08T08:28:33.736003+0000","last_became_active":"2024-03-08T08:28:31.887908+0000","last_became_peered":"2024-03-08T08:28:31.887908+0000","last_unstale":"2024-03-08T08:28:33.736003+0000","last_undegraded":"2024-03-08T08:28:33.736003+0000","last_fullsized":"2024-03-08T08:28:33.736003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:38:42.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735323+0000","last_change":"2024-03-08T08:28:31.892751+0000","last_active":"2024-03-08T08:28:33.735323+0000","last_peered":"2024-03-08T08:28:33.735323+0000","last_clean":"2024-03-08T08:28:33.735323+0000","last_became_active":"2024-03-08T08:28:31.892493+0000","last_became_peered":"2024-03-08T08:28:31.892493+0000","last_unstale":"2024-03-08T08:28:33.735323+0000","last_undegraded":"2024-03-08T08:28:33.735323+0000","last_fullsized":"2024-03-08T08:28:33.735323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:25:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883969+0000","last_change":"2024-03-08T08:28:31.885989+0000","last_active":"2024-03-08T08:28:32.883969+0000","last_peered":"2024-03-08T08:28:32.883969+0000","last_clean":"2024-03-08T08:28:32.883969+0000","last_became_active":"2024-03-08T08:28:31.885404+0000","last_became_peered":"2024-03-08T08:28:31.885404+0000","last_unstale":"2024-03-08T08:28:32.883969+0000","last_undegraded":"2024-03-08T08:28:32.883969+0000","last_fullsized":"2024-03-08T08:28:32.883969+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:48:19.249384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883381+0000","last_change":"2024-03-08T08:28:31.883948+0000","last_active":"2024-03-08T08:28:32.883381+0000","last_peered":"2024-03-08T08:28:32.883381+0000","last_clean":"2024-03-08T08:28:32.883381+0000","last_became_active":"2024-03-08T08:28:31.883713+0000","last_became_peered":"2024-03-08T08:28:31.883713+0000","last_unstale":"2024-03-08T08:28:32.883381+0000","last_undegraded":"2024-03-08T08:28:32.883381+0000","last_fullsized":"2024-03-08T08:28:32.883381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:04:20.429026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735979+0000","last_change":"2024-03-08T08:28:31.885505+0000","last_active":"2024-03-08T08:28:33.735979+0000","last_peered":"2024-03-08T08:28:33.735979+0000","last_clean":"2024-03-08T08:28:33.735979+0000","last_became_active":"2024-03-08T08:28:31.885222+0000","last_became_peered":"2024-03-08T08:28:31.885222+0000","last_unstale":"2024-03-08T08:28:33.735979+0000","last_undegraded":"2024-03-08T08:28:33.735979+0000","last_fullsized":"2024-03-08T08:28:33.735979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:45.259078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735692+0000","last_change":"2024-03-08T08:28:31.887897+0000","last_active":"2024-03-08T08:28:33.735692+0000","last_peered":"2024-03-08T08:28:33.735692+0000","last_clean":"2024-03-08T08:28:33.735692+0000","last_became_active":"2024-03-08T08:28:31.887460+0000","last_became_peered":"2024-03-08T08:28:31.887460+0000","last_unstale":"2024-03-08T08:28:33.735692+0000","last_undegraded":"2024-03-08T08:28:33.735692+0000","last_fullsized":"2024-03-08T08:28:33.735692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:04:20.429026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735417+0000","last_change":"2024-03-08T08:28:31.889738+0000","last_active":"2024-03-08T08:28:33.735417+0000","last_peered":"2024-03-08T08:28:33.735417+0000","last_clean":"2024-03-08T08:28:33.735417+0000","last_became_active":"2024-03-08T08:28:31.889459+0000","last_became_peered":"2024-03-08T08:28:31.889459+0000","last_unstale":"2024-03-08T08:28:33.735417+0000","last_undegraded":"2024-03-08T08:28:33.735417+0000","last_fullsized":"2024-03-08T08:28:33.735417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:25:08.892782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735635+0000","last_change":"2024-03-08T08:28:31.880370+0000","last_active":"2024-03-08T08:28:33.735635+0000","last_peered":"2024-03-08T08:28:33.735635+0000","last_clean":"2024-03-08T08:28:33.735635+0000","last_became_active":"2024-03-08T08:28:31.880076+0000","last_became_peered":"2024-03-08T08:28:31.880076+0000","last_unstale":"2024-03-08T08:28:33.735635+0000","last_undegraded":"2024-03-08T08:28:33.735635+0000","last_fullsized":"2024-03-08T08:28:33.735635+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:18:24.097031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735880+0000","last_change":"2024-03-08T08:28:31.883664+0000","last_active":"2024-03-08T08:28:33.735880+0000","last_peered":"2024-03-08T08:28:33.735880+0000","last_clean":"2024-03-08T08:28:33.735880+0000","last_became_active":"2024-03-08T08:28:31.883233+0000","last_became_peered":"2024-03-08T08:28:31.883233+0000","last_unstale":"2024-03-08T08:28:33.735880+0000","last_undegraded":"2024-03-08T08:28:33.735880+0000","last_fullsized":"2024-03-08T08:28:33.735880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:38:42.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736169+0000","last_change":"2024-03-08T08:28:31.893080+0000","last_active":"2024-03-08T08:28:33.736169+0000","last_peered":"2024-03-08T08:28:33.736169+0000","last_clean":"2024-03-08T08:28:33.736169+0000","last_became_active":"2024-03-08T08:28:31.892764+0000","last_became_peered":"2024-03-08T08:28:31.892764+0000","last_unstale":"2024-03-08T08:28:33.736169+0000","last_undegraded":"2024-03-08T08:28:33.736169+0000","last_fullsized":"2024-03-08T08:28:33.736169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:12:32.298587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735679+0000","last_change":"2024-03-08T08:28:31.894358+0000","last_active":"2024-03-08T08:28:33.735679+0000","last_peered":"2024-03-08T08:28:33.735679+0000","last_clean":"2024-03-08T08:28:33.735679+0000","last_became_active":"2024-03-08T08:28:31.894080+0000","last_became_peered":"2024-03-08T08:28:31.894080+0000","last_unstale":"2024-03-08T08:28:33.735679+0000","last_undegraded":"2024-03-08T08:28:33.735679+0000","last_fullsized":"2024-03-08T08:28:33.735679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:04:20.429026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736139+0000","last_change":"2024-03-08T08:28:31.886504+0000","last_active":"2024-03-08T08:28:33.736139+0000","last_peered":"2024-03-08T08:28:33.736139+0000","last_clean":"2024-03-08T08:28:33.736139+0000","last_became_active":"2024-03-08T08:28:31.886244+0000","last_became_peered":"2024-03-08T08:28:31.886244+0000","last_unstale":"2024-03-08T08:28:33.736139+0000","last_undegraded":"2024-03-08T08:28:33.736139+0000","last_fullsized":"2024-03-08T08:28:33.736139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:25:39.898079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735868+0000","last_change":"2024-03-08T08:28:31.887937+0000","last_active":"2024-03-08T08:28:33.735868+0000","last_peered":"2024-03-08T08:28:33.735868+0000","last_clean":"2024-03-08T08:28:33.735868+0000","last_became_active":"2024-03-08T08:28:31.887691+0000","last_became_peered":"2024-03-08T08:28:31.887691+0000","last_unstale":"2024-03-08T08:28:33.735868+0000","last_undegraded":"2024-03-08T08:28:33.735868+0000","last_fullsized":"2024-03-08T08:28:33.735868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:15:02.135580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735318+0000","last_change":"2024-03-08T08:28:31.884064+0000","last_active":"2024-03-08T08:28:33.735318+0000","last_peered":"2024-03-08T08:28:33.735318+0000","last_clean":"2024-03-08T08:28:33.735318+0000","last_became_active":"2024-03-08T08:28:31.883815+0000","last_became_peered":"2024-03-08T08:28:31.883815+0000","last_unstale":"2024-03-08T08:28:33.735318+0000","last_undegraded":"2024-03-08T08:28:33.735318+0000","last_fullsized":"2024-03-08T08:28:33.735318+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:00:56.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884861+0000","last_change":"2024-03-08T08:28:31.897247+0000","last_active":"2024-03-08T08:28:32.884861+0000","last_peered":"2024-03-08T08:28:32.884861+0000","last_clean":"2024-03-08T08:28:32.884861+0000","last_became_active":"2024-03-08T08:28:31.897011+0000","last_became_peered":"2024-03-08T08:28:31.897011+0000","last_unstale":"2024-03-08T08:28:32.884861+0000","last_undegraded":"2024-03-08T08:28:32.884861+0000","last_fullsized":"2024-03-08T08:28:32.884861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:46:10.595778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884770+0000","last_change":"2024-03-08T08:28:31.885404+0000","last_active":"2024-03-08T08:28:32.884770+0000","last_peered":"2024-03-08T08:28:32.884770+0000","last_clean":"2024-03-08T08:28:32.884770+0000","last_became_active":"2024-03-08T08:28:31.885124+0000","last_became_peered":"2024-03-08T08:28:31.885124+0000","last_unstale":"2024-03-08T08:28:32.884770+0000","last_undegraded":"2024-03-08T08:28:32.884770+0000","last_fullsized":"2024-03-08T08:28:32.884770+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:43:45.842491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886447+0000","last_change":"2024-03-08T08:28:31.895966+0000","last_active":"2024-03-08T08:28:32.886447+0000","last_peered":"2024-03-08T08:28:32.886447+0000","last_clean":"2024-03-08T08:28:32.886447+0000","last_became_active":"2024-03-08T08:28:31.895563+0000","last_became_peered":"2024-03-08T08:28:31.895563+0000","last_unstale":"2024-03-08T08:28:32.886447+0000","last_undegraded":"2024-03-08T08:28:32.886447+0000","last_fullsized":"2024-03-08T08:28:32.886447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:02.761073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735803+0000","last_change":"2024-03-08T08:28:31.880375+0000","last_active":"2024-03-08T08:28:33.735803+0000","last_peered":"2024-03-08T08:28:33.735803+0000","last_clean":"2024-03-08T08:28:33.735803+0000","last_became_active":"2024-03-08T08:28:31.880087+0000","last_became_peered":"2024-03-08T08:28:31.880087+0000","last_unstale":"2024-03-08T08:28:33.735803+0000","last_undegraded":"2024-03-08T08:28:33.735803+0000","last_fullsized":"2024-03-08T08:28:33.735803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:37:57.838372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883360+0000","last_change":"2024-03-08T08:28:31.890946+0000","last_active":"2024-03-08T08:28:32.883360+0000","last_peered":"2024-03-08T08:28:32.883360+0000","last_clean":"2024-03-08T08:28:32.883360+0000","last_became_active":"2024-03-08T08:28:31.890385+0000","last_became_peered":"2024-03-08T08:28:31.890385+0000","last_unstale":"2024-03-08T08:28:32.883360+0000","last_undegraded":"2024-03-08T08:28:32.883360+0000","last_fullsized":"2024-03-08T08:28:32.883360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:59:31.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736157+0000","last_change":"2024-03-08T08:28:31.888448+0000","last_active":"2024-03-08T08:28:33.736157+0000","last_peered":"2024-03-08T08:28:33.736157+0000","last_clean":"2024-03-08T08:28:33.736157+0000","last_became_active":"2024-03-08T08:28:31.888165+0000","last_became_peered":"2024-03-08T08:28:31.888165+0000","last_unstale":"2024-03-08T08:28:33.736157+0000","last_undegraded":"2024-03-08T08:28:33.736157+0000","last_fullsized":"2024-03-08T08:28:33.736157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:42:31.817879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883308+0000","last_change":"2024-03-08T08:28:31.892838+0000","last_active":"2024-03-08T08:28:32.883308+0000","last_peered":"2024-03-08T08:28:32.883308+0000","last_clean":"2024-03-08T08:28:32.883308+0000","last_became_active":"2024-03-08T08:28:31.890655+0000","last_became_peered":"2024-03-08T08:28:31.890655+0000","last_unstale":"2024-03-08T08:28:32.883308+0000","last_undegraded":"2024-03-08T08:28:32.883308+0000","last_fullsized":"2024-03-08T08:28:32.883308+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:25:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735985+0000","last_change":"2024-03-08T08:28:31.896631+0000","last_active":"2024-03-08T08:28:33.735985+0000","last_peered":"2024-03-08T08:28:33.735985+0000","last_clean":"2024-03-08T08:28:33.735985+0000","last_became_active":"2024-03-08T08:28:31.896366+0000","last_became_peered":"2024-03-08T08:28:31.896366+0000","last_unstale":"2024-03-08T08:28:33.735985+0000","last_undegraded":"2024-03-08T08:28:33.735985+0000","last_fullsized":"2024-03-08T08:28:33.735985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:45:58.883654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884957+0000","last_change":"2024-03-08T08:28:31.889706+0000","last_active":"2024-03-08T08:28:32.884957+0000","last_peered":"2024-03-08T08:28:32.884957+0000","last_clean":"2024-03-08T08:28:32.884957+0000","last_became_active":"2024-03-08T08:28:31.889432+0000","last_became_peered":"2024-03-08T08:28:31.889432+0000","last_unstale":"2024-03-08T08:28:32.884957+0000","last_undegraded":"2024-03-08T08:28:32.884957+0000","last_fullsized":"2024-03-08T08:28:32.884957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:00:17.122569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735718+0000","last_change":"2024-03-08T08:28:31.886354+0000","last_active":"2024-03-08T08:28:33.735718+0000","last_peered":"2024-03-08T08:28:33.735718+0000","last_clean":"2024-03-08T08:28:33.735718+0000","last_became_active":"2024-03-08T08:28:31.886090+0000","last_became_peered":"2024-03-08T08:28:31.886090+0000","last_unstale":"2024-03-08T08:28:33.735718+0000","last_undegraded":"2024-03-08T08:28:33.735718+0000","last_fullsized":"2024-03-08T08:28:33.735718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:45:58.883654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884466+0000","last_change":"2024-03-08T08:28:31.882378+0000","last_active":"2024-03-08T08:28:32.884466+0000","last_peered":"2024-03-08T08:28:32.884466+0000","last_clean":"2024-03-08T08:28:32.884466+0000","last_became_active":"2024-03-08T08:28:31.881998+0000","last_became_peered":"2024-03-08T08:28:31.881998+0000","last_unstale":"2024-03-08T08:28:32.884466+0000","last_undegraded":"2024-03-08T08:28:32.884466+0000","last_fullsized":"2024-03-08T08:28:32.884466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:43:45.842491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735330+0000","last_change":"2024-03-08T08:28:31.891877+0000","last_active":"2024-03-08T08:28:33.735330+0000","last_peered":"2024-03-08T08:28:33.735330+0000","last_clean":"2024-03-08T08:28:33.735330+0000","last_became_active":"2024-03-08T08:28:31.891175+0000","last_became_peered":"2024-03-08T08:28:31.891175+0000","last_unstale":"2024-03-08T08:28:33.735330+0000","last_undegraded":"2024-03-08T08:28:33.735330+0000","last_fullsized":"2024-03-08T08:28:33.735330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:07:06.048301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886742+0000","last_change":"2024-03-08T08:28:31.896036+0000","last_active":"2024-03-08T08:28:32.886742+0000","last_peered":"2024-03-08T08:28:32.886742+0000","last_clean":"2024-03-08T08:28:32.886742+0000","last_became_active":"2024-03-08T08:28:31.895686+0000","last_became_peered":"2024-03-08T08:28:31.895686+0000","last_unstale":"2024-03-08T08:28:32.886742+0000","last_undegraded":"2024-03-08T08:28:32.886742+0000","last_fullsized":"2024-03-08T08:28:32.886742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:00:17.122569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884263+0000","last_change":"2024-03-08T08:28:30.869323+0000","last_active":"2024-03-08T08:28:32.884263+0000","last_peered":"2024-03-08T08:28:32.884263+0000","last_clean":"2024-03-08T08:28:32.884263+0000","last_became_active":"2024-03-08T08:28:30.869040+0000","last_became_peered":"2024-03-08T08:28:30.869040+0000","last_unstale":"2024-03-08T08:28:32.884263+0000","last_undegraded":"2024-03-08T08:28:32.884263+0000","last_fullsized":"2024-03-08T08:28:32.884263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:25:07.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884155+0000","last_change":"2024-03-08T08:28:31.886797+0000","last_active":"2024-03-08T08:28:32.884155+0000","last_peered":"2024-03-08T08:28:32.884155+0000","last_clean":"2024-03-08T08:28:32.884155+0000","last_became_active":"2024-03-08T08:28:31.886349+0000","last_became_peered":"2024-03-08T08:28:31.886349+0000","last_unstale":"2024-03-08T08:28:32.884155+0000","last_undegraded":"2024-03-08T08:28:32.884155+0000","last_fullsized":"2024-03-08T08:28:32.884155+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:02.761073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886690+0000","last_change":"2024-03-08T08:28:30.880963+0000","last_active":"2024-03-08T08:28:32.886690+0000","last_peered":"2024-03-08T08:28:32.886690+0000","last_clean":"2024-03-08T08:28:32.886690+0000","last_became_active":"2024-03-08T08:28:30.880613+0000","last_became_peered":"2024-03-08T08:28:30.880613+0000","last_unstale":"2024-03-08T08:28:32.886690+0000","last_undegraded":"2024-03-08T08:28:32.886690+0000","last_fullsized":"2024-03-08T08:28:32.886690+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:05:56.384648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884325+0000","last_change":"2024-03-08T08:28:31.887173+0000","last_active":"2024-03-08T08:28:32.884325+0000","last_peered":"2024-03-08T08:28:32.884325+0000","last_clean":"2024-03-08T08:28:32.884325+0000","last_became_active":"2024-03-08T08:28:31.885284+0000","last_became_peered":"2024-03-08T08:28:31.885284+0000","last_unstale":"2024-03-08T08:28:32.884325+0000","last_undegraded":"2024-03-08T08:28:32.884325+0000","last_fullsized":"2024-03-08T08:28:32.884325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:10:14.609573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735825+0000","last_change":"2024-03-08T08:28:30.877050+0000","last_active":"2024-03-08T08:28:33.735825+0000","last_peered":"2024-03-08T08:28:33.735825+0000","last_clean":"2024-03-08T08:28:33.735825+0000","last_became_active":"2024-03-08T08:28:30.876755+0000","last_became_peered":"2024-03-08T08:28:30.876755+0000","last_unstale":"2024-03-08T08:28:33.735825+0000","last_undegraded":"2024-03-08T08:28:33.735825+0000","last_fullsized":"2024-03-08T08:28:33.735825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:47:14.118244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883329+0000","last_change":"2024-03-08T08:28:31.884948+0000","last_active":"2024-03-08T08:28:32.883329+0000","last_peered":"2024-03-08T08:28:32.883329+0000","last_clean":"2024-03-08T08:28:32.883329+0000","last_became_active":"2024-03-08T08:28:31.884600+0000","last_became_peered":"2024-03-08T08:28:31.884600+0000","last_unstale":"2024-03-08T08:28:32.883329+0000","last_undegraded":"2024-03-08T08:28:32.883329+0000","last_fullsized":"2024-03-08T08:28:32.883329+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:07:06.048301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735401+0000","last_change":"2024-03-08T08:28:30.873806+0000","last_active":"2024-03-08T08:28:33.735401+0000","last_peered":"2024-03-08T08:28:33.735401+0000","last_clean":"2024-03-08T08:28:33.735401+0000","last_became_active":"2024-03-08T08:28:30.873361+0000","last_became_peered":"2024-03-08T08:28:30.873361+0000","last_unstale":"2024-03-08T08:28:33.735401+0000","last_undegraded":"2024-03-08T08:28:33.735401+0000","last_fullsized":"2024-03-08T08:28:33.735401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:58:56.785346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736131+0000","last_change":"2024-03-08T08:28:31.890672+0000","last_active":"2024-03-08T08:28:33.736131+0000","last_peered":"2024-03-08T08:28:33.736131+0000","last_clean":"2024-03-08T08:28:33.736131+0000","last_became_active":"2024-03-08T08:28:31.890084+0000","last_became_peered":"2024-03-08T08:28:31.890084+0000","last_unstale":"2024-03-08T08:28:33.736131+0000","last_undegraded":"2024-03-08T08:28:33.736131+0000","last_fullsized":"2024-03-08T08:28:33.736131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:08:07.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884971+0000","last_change":"2024-03-08T08:28:30.876030+0000","last_active":"2024-03-08T08:28:32.884971+0000","last_peered":"2024-03-08T08:28:32.884971+0000","last_clean":"2024-03-08T08:28:32.884971+0000","last_became_active":"2024-03-08T08:28:30.875703+0000","last_became_peered":"2024-03-08T08:28:30.875703+0000","last_unstale":"2024-03-08T08:28:32.884971+0000","last_undegraded":"2024-03-08T08:28:32.884971+0000","last_fullsized":"2024-03-08T08:28:32.884971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:40:51.920470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735632+0000","last_change":"2024-03-08T08:28:31.897889+0000","last_active":"2024-03-08T08:28:33.735632+0000","last_peered":"2024-03-08T08:28:33.735632+0000","last_clean":"2024-03-08T08:28:33.735632+0000","last_became_active":"2024-03-08T08:28:31.897571+0000","last_became_peered":"2024-03-08T08:28:31.897571+0000","last_unstale":"2024-03-08T08:28:33.735632+0000","last_undegraded":"2024-03-08T08:28:33.735632+0000","last_fullsized":"2024-03-08T08:28:33.735632+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:59:31.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884739+0000","last_change":"2024-03-08T08:28:30.873717+0000","last_active":"2024-03-08T08:28:32.884739+0000","last_peered":"2024-03-08T08:28:32.884739+0000","last_clean":"2024-03-08T08:28:32.884739+0000","last_became_active":"2024-03-08T08:28:30.873429+0000","last_became_peered":"2024-03-08T08:28:30.873429+0000","last_unstale":"2024-03-08T08:28:32.884739+0000","last_undegraded":"2024-03-08T08:28:32.884739+0000","last_fullsized":"2024-03-08T08:28:32.884739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:19.606713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883791+0000","last_change":"2024-03-08T08:28:31.887974+0000","last_active":"2024-03-08T08:28:32.883791+0000","last_peered":"2024-03-08T08:28:32.883791+0000","last_clean":"2024-03-08T08:28:32.883791+0000","last_became_active":"2024-03-08T08:28:31.887688+0000","last_became_peered":"2024-03-08T08:28:31.887688+0000","last_unstale":"2024-03-08T08:28:32.883791+0000","last_undegraded":"2024-03-08T08:28:32.883791+0000","last_fullsized":"2024-03-08T08:28:32.883791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:06:03.648335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735469+0000","last_change":"2024-03-08T08:28:30.877804+0000","last_active":"2024-03-08T08:28:33.735469+0000","last_peered":"2024-03-08T08:28:33.735469+0000","last_clean":"2024-03-08T08:28:33.735469+0000","last_became_active":"2024-03-08T08:28:30.877522+0000","last_became_peered":"2024-03-08T08:28:30.877522+0000","last_unstale":"2024-03-08T08:28:33.735469+0000","last_undegraded":"2024-03-08T08:28:33.735469+0000","last_fullsized":"2024-03-08T08:28:33.735469+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:01.757047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.882991+0000","last_change":"2024-03-08T08:28:31.884482+0000","last_active":"2024-03-08T08:28:32.882991+0000","last_peered":"2024-03-08T08:28:32.882991+0000","last_clean":"2024-03-08T08:28:32.882991+0000","last_became_active":"2024-03-08T08:28:31.884239+0000","last_became_peered":"2024-03-08T08:28:31.884239+0000","last_unstale":"2024-03-08T08:28:32.882991+0000","last_undegraded":"2024-03-08T08:28:32.882991+0000","last_fullsized":"2024-03-08T08:28:32.882991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:36:39.113162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884502+0000","last_change":"2024-03-08T08:28:30.869588+0000","last_active":"2024-03-08T08:28:32.884502+0000","last_peered":"2024-03-08T08:28:32.884502+0000","last_clean":"2024-03-08T08:28:32.884502+0000","last_became_active":"2024-03-08T08:28:30.869274+0000","last_became_peered":"2024-03-08T08:28:30.869274+0000","last_unstale":"2024-03-08T08:28:32.884502+0000","last_undegraded":"2024-03-08T08:28:32.884502+0000","last_fullsized":"2024-03-08T08:28:32.884502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:27:42.196078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736182+0000","last_change":"2024-03-08T08:28:31.883589+0000","last_active":"2024-03-08T08:28:33.736182+0000","last_peered":"2024-03-08T08:28:33.736182+0000","last_clean":"2024-03-08T08:28:33.736182+0000","last_became_active":"2024-03-08T08:28:31.883117+0000","last_became_peered":"2024-03-08T08:28:31.883117+0000","last_unstale":"2024-03-08T08:28:33.736182+0000","last_undegraded":"2024-03-08T08:28:33.736182+0000","last_fullsized":"2024-03-08T08:28:33.736182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:18:24.097031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736078+0000","last_change":"2024-03-08T08:28:30.881215+0000","last_active":"2024-03-08T08:28:33.736078+0000","last_peered":"2024-03-08T08:28:33.736078+0000","last_clean":"2024-03-08T08:28:33.736078+0000","last_became_active":"2024-03-08T08:28:30.880940+0000","last_became_peered":"2024-03-08T08:28:30.880940+0000","last_unstale":"2024-03-08T08:28:33.736078+0000","last_undegraded":"2024-03-08T08:28:33.736078+0000","last_fullsized":"2024-03-08T08:28:33.736078+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:40:29.600871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736067+0000","last_change":"2024-03-08T08:28:31.886487+0000","last_active":"2024-03-08T08:28:33.736067+0000","last_peered":"2024-03-08T08:28:33.736067+0000","last_clean":"2024-03-08T08:28:33.736067+0000","last_became_active":"2024-03-08T08:28:31.886226+0000","last_became_peered":"2024-03-08T08:28:31.886226+0000","last_unstale":"2024-03-08T08:28:33.736067+0000","last_undegraded":"2024-03-08T08:28:33.736067+0000","last_fullsized":"2024-03-08T08:28:33.736067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:12:31.649105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886836+0000","last_change":"2024-03-08T08:28:30.876384+0000","last_active":"2024-03-08T08:28:32.886836+0000","last_peered":"2024-03-08T08:28:32.886836+0000","last_clean":"2024-03-08T08:28:32.886836+0000","last_became_active":"2024-03-08T08:28:30.876053+0000","last_became_peered":"2024-03-08T08:28:30.876053+0000","last_unstale":"2024-03-08T08:28:32.886836+0000","last_undegraded":"2024-03-08T08:28:32.886836+0000","last_fullsized":"2024-03-08T08:28:32.886836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:25:07.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735872+0000","last_change":"2024-03-08T08:28:31.887577+0000","last_active":"2024-03-08T08:28:33.735872+0000","last_peered":"2024-03-08T08:28:33.735872+0000","last_clean":"2024-03-08T08:28:33.735872+0000","last_became_active":"2024-03-08T08:28:31.887101+0000","last_became_peered":"2024-03-08T08:28:31.887101+0000","last_unstale":"2024-03-08T08:28:33.735872+0000","last_undegraded":"2024-03-08T08:28:33.735872+0000","last_fullsized":"2024-03-08T08:28:33.735872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:49:58.833028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736267+0000","last_change":"2024-03-08T08:28:30.880920+0000","last_active":"2024-03-08T08:28:33.736267+0000","last_peered":"2024-03-08T08:28:33.736267+0000","last_clean":"2024-03-08T08:28:33.736267+0000","last_became_active":"2024-03-08T08:28:30.880573+0000","last_became_peered":"2024-03-08T08:28:30.880573+0000","last_unstale":"2024-03-08T08:28:33.736267+0000","last_undegraded":"2024-03-08T08:28:33.736267+0000","last_fullsized":"2024-03-08T08:28:33.736267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:48:18.245357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735496+0000","last_change":"2024-03-08T08:28:31.891159+0000","last_active":"2024-03-08T08:28:33.735496+0000","last_peered":"2024-03-08T08:28:33.735496+0000","last_clean":"2024-03-08T08:28:33.735496+0000","last_became_active":"2024-03-08T08:28:31.890826+0000","last_became_peered":"2024-03-08T08:28:31.890826+0000","last_unstale":"2024-03-08T08:28:33.735496+0000","last_undegraded":"2024-03-08T08:28:33.735496+0000","last_fullsized":"2024-03-08T08:28:33.735496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:08:07.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735829+0000","last_change":"2024-03-08T08:28:30.879905+0000","last_active":"2024-03-08T08:28:33.735829+0000","last_peered":"2024-03-08T08:28:33.735829+0000","last_clean":"2024-03-08T08:28:33.735829+0000","last_became_active":"2024-03-08T08:28:30.879646+0000","last_became_peered":"2024-03-08T08:28:30.879646+0000","last_unstale":"2024-03-08T08:28:33.735829+0000","last_undegraded":"2024-03-08T08:28:33.735829+0000","last_fullsized":"2024-03-08T08:28:33.735829+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:00:16.118543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883437+0000","last_change":"2024-03-08T08:28:31.883574+0000","last_active":"2024-03-08T08:28:32.883437+0000","last_peered":"2024-03-08T08:28:32.883437+0000","last_clean":"2024-03-08T08:28:32.883437+0000","last_became_active":"2024-03-08T08:28:31.883307+0000","last_became_peered":"2024-03-08T08:28:31.883307+0000","last_unstale":"2024-03-08T08:28:32.883437+0000","last_undegraded":"2024-03-08T08:28:32.883437+0000","last_fullsized":"2024-03-08T08:28:32.883437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:37:57.838372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.885062+0000","last_change":"2024-03-08T08:28:30.869336+0000","last_active":"2024-03-08T08:28:32.885062+0000","last_peered":"2024-03-08T08:28:32.885062+0000","last_clean":"2024-03-08T08:28:32.885062+0000","last_became_active":"2024-03-08T08:28:30.868819+0000","last_became_peered":"2024-03-08T08:28:30.868819+0000","last_unstale":"2024-03-08T08:28:32.885062+0000","last_undegraded":"2024-03-08T08:28:32.885062+0000","last_fullsized":"2024-03-08T08:28:32.885062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:36:30.895469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735782+0000","last_change":"2024-03-08T08:28:31.898210+0000","last_active":"2024-03-08T08:28:33.735782+0000","last_peered":"2024-03-08T08:28:33.735782+0000","last_clean":"2024-03-08T08:28:33.735782+0000","last_became_active":"2024-03-08T08:28:31.897868+0000","last_became_peered":"2024-03-08T08:28:31.897868+0000","last_unstale":"2024-03-08T08:28:33.735782+0000","last_undegraded":"2024-03-08T08:28:33.735782+0000","last_fullsized":"2024-03-08T08:28:33.735782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T20:15:34.028975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883582+0000","last_change":"2024-03-08T08:28:30.866735+0000","last_active":"2024-03-08T08:28:32.883582+0000","last_peered":"2024-03-08T08:28:32.883582+0000","last_clean":"2024-03-08T08:28:32.883582+0000","last_became_active":"2024-03-08T08:28:30.866479+0000","last_became_peered":"2024-03-08T08:28:30.866479+0000","last_unstale":"2024-03-08T08:28:32.883582+0000","last_undegraded":"2024-03-08T08:28:32.883582+0000","last_fullsized":"2024-03-08T08:28:32.883582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:02.666304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735505+0000","last_change":"2024-03-08T08:28:31.892179+0000","last_active":"2024-03-08T08:28:33.735505+0000","last_peered":"2024-03-08T08:28:33.735505+0000","last_clean":"2024-03-08T08:28:33.735505+0000","last_became_active":"2024-03-08T08:28:31.891416+0000","last_became_peered":"2024-03-08T08:28:31.891416+0000","last_unstale":"2024-03-08T08:28:33.735505+0000","last_undegraded":"2024-03-08T08:28:33.735505+0000","last_fullsized":"2024-03-08T08:28:33.735505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:56:32.913207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735573+0000","last_change":"2024-03-08T08:28:30.881378+0000","last_active":"2024-03-08T08:28:33.735573+0000","last_peered":"2024-03-08T08:28:33.735573+0000","last_clean":"2024-03-08T08:28:33.735573+0000","last_became_active":"2024-03-08T08:28:30.880959+0000","last_became_peered":"2024-03-08T08:28:30.880959+0000","last_unstale":"2024-03-08T08:28:33.735573+0000","last_undegraded":"2024-03-08T08:28:33.735573+0000","last_fullsized":"2024-03-08T08:28:33.735573+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:46:09.591752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884624+0000","last_change":"2024-03-08T08:28:31.882245+0000","last_active":"2024-03-08T08:28:32.884624+0000","last_peered":"2024-03-08T08:28:32.884624+0000","last_clean":"2024-03-08T08:28:32.884624+0000","last_became_active":"2024-03-08T08:28:31.881658+0000","last_became_peered":"2024-03-08T08:28:31.881658+0000","last_unstale":"2024-03-08T08:28:32.884624+0000","last_undegraded":"2024-03-08T08:28:32.884624+0000","last_fullsized":"2024-03-08T08:28:32.884624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:46:10.595778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735133+0000","last_change":"2024-03-08T08:28:30.875030+0000","last_active":"2024-03-08T08:28:33.735133+0000","last_peered":"2024-03-08T08:28:33.735133+0000","last_clean":"2024-03-08T08:28:33.735133+0000","last_became_active":"2024-03-08T08:28:30.874753+0000","last_became_peered":"2024-03-08T08:28:30.874753+0000","last_unstale":"2024-03-08T08:28:33.735133+0000","last_undegraded":"2024-03-08T08:28:33.735133+0000","last_fullsized":"2024-03-08T08:28:33.735133+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:10:13.605547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884618+0000","last_change":"2024-03-08T08:28:31.886559+0000","last_active":"2024-03-08T08:28:32.884618+0000","last_peered":"2024-03-08T08:28:32.884618+0000","last_clean":"2024-03-08T08:28:32.884618+0000","last_became_active":"2024-03-08T08:28:31.885331+0000","last_became_peered":"2024-03-08T08:28:31.885331+0000","last_unstale":"2024-03-08T08:28:32.884618+0000","last_undegraded":"2024-03-08T08:28:32.884618+0000","last_fullsized":"2024-03-08T08:28:32.884618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:51:02.761073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883523+0000","last_change":"2024-03-08T08:28:30.873585+0000","last_active":"2024-03-08T08:28:32.883523+0000","last_peered":"2024-03-08T08:28:32.883523+0000","last_clean":"2024-03-08T08:28:32.883523+0000","last_became_active":"2024-03-08T08:28:30.873290+0000","last_became_peered":"2024-03-08T08:28:30.873290+0000","last_unstale":"2024-03-08T08:28:32.883523+0000","last_undegraded":"2024-03-08T08:28:32.883523+0000","last_fullsized":"2024-03-08T08:28:32.883523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T12:05:57.997987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735694+0000","last_change":"2024-03-08T08:28:31.888083+0000","last_active":"2024-03-08T08:28:33.735694+0000","last_peered":"2024-03-08T08:28:33.735694+0000","last_clean":"2024-03-08T08:28:33.735694+0000","last_became_active":"2024-03-08T08:28:31.887706+0000","last_became_peered":"2024-03-08T08:28:31.887706+0000","last_unstale":"2024-03-08T08:28:33.735694+0000","last_undegraded":"2024-03-08T08:28:33.735694+0000","last_fullsized":"2024-03-08T08:28:33.735694+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:56:32.913207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884721+0000","last_change":"2024-03-08T08:28:30.871544+0000","last_active":"2024-03-08T08:28:32.884721+0000","last_peered":"2024-03-08T08:28:32.884721+0000","last_clean":"2024-03-08T08:28:32.884721+0000","last_became_active":"2024-03-08T08:28:30.871236+0000","last_became_peered":"2024-03-08T08:28:30.871236+0000","last_unstale":"2024-03-08T08:28:32.884721+0000","last_undegraded":"2024-03-08T08:28:32.884721+0000","last_fullsized":"2024-03-08T08:28:32.884721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:02:51.963302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883352+0000","last_change":"2024-03-08T08:28:31.892506+0000","last_active":"2024-03-08T08:28:32.883352+0000","last_peered":"2024-03-08T08:28:32.883352+0000","last_clean":"2024-03-08T08:28:32.883352+0000","last_became_active":"2024-03-08T08:28:31.890721+0000","last_became_peered":"2024-03-08T08:28:31.890721+0000","last_unstale":"2024-03-08T08:28:32.883352+0000","last_undegraded":"2024-03-08T08:28:32.883352+0000","last_fullsized":"2024-03-08T08:28:32.883352+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:56:32.913207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884415+0000","last_change":"2024-03-08T08:28:30.869691+0000","last_active":"2024-03-08T08:28:32.884415+0000","last_peered":"2024-03-08T08:28:32.884415+0000","last_clean":"2024-03-08T08:28:32.884415+0000","last_became_active":"2024-03-08T08:28:30.869367+0000","last_became_peered":"2024-03-08T08:28:30.869367+0000","last_unstale":"2024-03-08T08:28:32.884415+0000","last_undegraded":"2024-03-08T08:28:32.884415+0000","last_fullsized":"2024-03-08T08:28:32.884415+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:06:55.899542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735562+0000","last_change":"2024-03-08T08:28:31.890286+0000","last_active":"2024-03-08T08:28:33.735562+0000","last_peered":"2024-03-08T08:28:33.735562+0000","last_clean":"2024-03-08T08:28:33.735562+0000","last_became_active":"2024-03-08T08:28:31.889565+0000","last_became_peered":"2024-03-08T08:28:31.889565+0000","last_unstale":"2024-03-08T08:28:33.735562+0000","last_undegraded":"2024-03-08T08:28:33.735562+0000","last_fullsized":"2024-03-08T08:28:33.735562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:18:00.495377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884425+0000","last_change":"2024-03-08T08:28:30.874068+0000","last_active":"2024-03-08T08:28:32.884425+0000","last_peered":"2024-03-08T08:28:32.884425+0000","last_clean":"2024-03-08T08:28:32.884425+0000","last_became_active":"2024-03-08T08:28:30.873748+0000","last_became_peered":"2024-03-08T08:28:30.873748+0000","last_unstale":"2024-03-08T08:28:32.884425+0000","last_undegraded":"2024-03-08T08:28:32.884425+0000","last_fullsized":"2024-03-08T08:28:32.884425+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:25:44.230931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735393+0000","last_change":"2024-03-08T08:28:31.888030+0000","last_active":"2024-03-08T08:28:33.735393+0000","last_peered":"2024-03-08T08:28:33.735393+0000","last_clean":"2024-03-08T08:28:33.735393+0000","last_became_active":"2024-03-08T08:28:31.887777+0000","last_became_peered":"2024-03-08T08:28:31.887777+0000","last_unstale":"2024-03-08T08:28:33.735393+0000","last_undegraded":"2024-03-08T08:28:33.735393+0000","last_fullsized":"2024-03-08T08:28:33.735393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:05:57.388674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886556+0000","last_change":"2024-03-08T08:28:30.880129+0000","last_active":"2024-03-08T08:28:32.886556+0000","last_peered":"2024-03-08T08:28:32.886556+0000","last_clean":"2024-03-08T08:28:32.886556+0000","last_became_active":"2024-03-08T08:28:30.879847+0000","last_became_peered":"2024-03-08T08:28:30.879847+0000","last_unstale":"2024-03-08T08:28:32.886556+0000","last_undegraded":"2024-03-08T08:28:32.886556+0000","last_fullsized":"2024-03-08T08:28:32.886556+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:02.666304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735578+0000","last_change":"2024-03-08T08:28:31.899285+0000","last_active":"2024-03-08T08:28:33.735578+0000","last_peered":"2024-03-08T08:28:33.735578+0000","last_clean":"2024-03-08T08:28:33.735578+0000","last_became_active":"2024-03-08T08:28:31.899018+0000","last_became_peered":"2024-03-08T08:28:31.899018+0000","last_unstale":"2024-03-08T08:28:33.735578+0000","last_undegraded":"2024-03-08T08:28:33.735578+0000","last_fullsized":"2024-03-08T08:28:33.735578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:51:03.670330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.884493+0000","last_change":"2024-03-08T08:28:30.866833+0000","last_active":"2024-03-08T08:28:32.884493+0000","last_peered":"2024-03-08T08:28:32.884493+0000","last_clean":"2024-03-08T08:28:32.884493+0000","last_became_active":"2024-03-08T08:28:30.866548+0000","last_became_peered":"2024-03-08T08:28:30.866548+0000","last_unstale":"2024-03-08T08:28:32.884493+0000","last_undegraded":"2024-03-08T08:28:32.884493+0000","last_fullsized":"2024-03-08T08:28:32.884493+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T18:05:56.384648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735225+0000","last_change":"2024-03-08T08:28:31.898760+0000","last_active":"2024-03-08T08:28:33.735225+0000","last_peered":"2024-03-08T08:28:33.735225+0000","last_clean":"2024-03-08T08:28:33.735225+0000","last_became_active":"2024-03-08T08:28:31.898468+0000","last_became_peered":"2024-03-08T08:28:31.898468+0000","last_unstale":"2024-03-08T08:28:33.735225+0000","last_undegraded":"2024-03-08T08:28:33.735225+0000","last_fullsized":"2024-03-08T08:28:33.735225+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:02:20.610739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.885094+0000","last_change":"2024-03-08T08:28:30.869452+0000","last_active":"2024-03-08T08:28:32.885094+0000","last_peered":"2024-03-08T08:28:32.885094+0000","last_clean":"2024-03-08T08:28:32.885094+0000","last_became_active":"2024-03-08T08:28:30.868730+0000","last_became_peered":"2024-03-08T08:28:30.868730+0000","last_unstale":"2024-03-08T08:28:32.885094+0000","last_undegraded":"2024-03-08T08:28:32.885094+0000","last_fullsized":"2024-03-08T08:28:32.885094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T13:02:17.692451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735456+0000","last_change":"2024-03-08T08:28:31.887577+0000","last_active":"2024-03-08T08:28:33.735456+0000","last_peered":"2024-03-08T08:28:33.735456+0000","last_clean":"2024-03-08T08:28:33.735456+0000","last_became_active":"2024-03-08T08:28:31.887285+0000","last_became_peered":"2024-03-08T08:28:31.887285+0000","last_unstale":"2024-03-08T08:28:33.735456+0000","last_undegraded":"2024-03-08T08:28:33.735456+0000","last_fullsized":"2024-03-08T08:28:33.735456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T10:59:31.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735312+0000","last_change":"2024-03-08T08:28:30.881897+0000","last_active":"2024-03-08T08:28:33.735312+0000","last_peered":"2024-03-08T08:28:33.735312+0000","last_clean":"2024-03-08T08:28:33.735312+0000","last_became_active":"2024-03-08T08:28:30.881553+0000","last_became_peered":"2024-03-08T08:28:30.881553+0000","last_unstale":"2024-03-08T08:28:33.735312+0000","last_undegraded":"2024-03-08T08:28:33.735312+0000","last_fullsized":"2024-03-08T08:28:33.735312+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:06:02.644309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735366+0000","last_change":"2024-03-08T08:28:31.883433+0000","last_active":"2024-03-08T08:28:33.735366+0000","last_peered":"2024-03-08T08:28:33.735366+0000","last_clean":"2024-03-08T08:28:33.735366+0000","last_became_active":"2024-03-08T08:28:31.883165+0000","last_became_peered":"2024-03-08T08:28:31.883165+0000","last_unstale":"2024-03-08T08:28:33.735366+0000","last_undegraded":"2024-03-08T08:28:33.735366+0000","last_fullsized":"2024-03-08T08:28:33.735366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:57.553847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883505+0000","last_change":"2024-03-08T08:28:30.869401+0000","last_active":"2024-03-08T08:28:32.883505+0000","last_peered":"2024-03-08T08:28:32.883505+0000","last_clean":"2024-03-08T08:28:32.883505+0000","last_became_active":"2024-03-08T08:28:30.869130+0000","last_became_peered":"2024-03-08T08:28:30.869130+0000","last_unstale":"2024-03-08T08:28:32.883505+0000","last_undegraded":"2024-03-08T08:28:32.883505+0000","last_fullsized":"2024-03-08T08:28:32.883505+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:43:10.394088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735861+0000","last_change":"2024-03-08T08:28:31.887190+0000","last_active":"2024-03-08T08:28:33.735861+0000","last_peered":"2024-03-08T08:28:33.735861+0000","last_clean":"2024-03-08T08:28:33.735861+0000","last_became_active":"2024-03-08T08:28:31.885851+0000","last_became_peered":"2024-03-08T08:28:31.885851+0000","last_unstale":"2024-03-08T08:28:33.735861+0000","last_undegraded":"2024-03-08T08:28:33.735861+0000","last_fullsized":"2024-03-08T08:28:33.735861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:55:45.259078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886830+0000","last_change":"2024-03-08T08:28:30.880219+0000","last_active":"2024-03-08T08:28:32.886830+0000","last_peered":"2024-03-08T08:28:32.886830+0000","last_clean":"2024-03-08T08:28:32.886830+0000","last_became_active":"2024-03-08T08:28:30.879954+0000","last_became_peered":"2024-03-08T08:28:30.879954+0000","last_unstale":"2024-03-08T08:28:32.886830+0000","last_undegraded":"2024-03-08T08:28:32.886830+0000","last_fullsized":"2024-03-08T08:28:32.886830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:17:59.491351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886581+0000","last_change":"2024-03-08T08:28:31.896128+0000","last_active":"2024-03-08T08:28:32.886581+0000","last_peered":"2024-03-08T08:28:32.886581+0000","last_clean":"2024-03-08T08:28:32.886581+0000","last_became_active":"2024-03-08T08:28:31.895841+0000","last_became_peered":"2024-03-08T08:28:31.895841+0000","last_unstale":"2024-03-08T08:28:32.886581+0000","last_undegraded":"2024-03-08T08:28:32.886581+0000","last_fullsized":"2024-03-08T08:28:32.886581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T15:46:10.595778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883475+0000","last_change":"2024-03-08T08:28:30.873467+0000","last_active":"2024-03-08T08:28:32.883475+0000","last_peered":"2024-03-08T08:28:32.883475+0000","last_clean":"2024-03-08T08:28:32.883475+0000","last_became_active":"2024-03-08T08:28:30.873184+0000","last_became_peered":"2024-03-08T08:28:30.873184+0000","last_unstale":"2024-03-08T08:28:32.883475+0000","last_undegraded":"2024-03-08T08:28:32.883475+0000","last_fullsized":"2024-03-08T08:28:32.883475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:06:02.644309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883958+0000","last_change":"2024-03-08T08:28:31.896662+0000","last_active":"2024-03-08T08:28:32.883958+0000","last_peered":"2024-03-08T08:28:32.883958+0000","last_clean":"2024-03-08T08:28:32.883958+0000","last_became_active":"2024-03-08T08:28:31.896347+0000","last_became_peered":"2024-03-08T08:28:31.896347+0000","last_unstale":"2024-03-08T08:28:32.883958+0000","last_undegraded":"2024-03-08T08:28:32.883958+0000","last_fullsized":"2024-03-08T08:28:32.883958+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T16:48:19.249384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886702+0000","last_change":"2024-03-08T08:28:30.880960+0000","last_active":"2024-03-08T08:28:32.886702+0000","last_peered":"2024-03-08T08:28:32.886702+0000","last_clean":"2024-03-08T08:28:32.886702+0000","last_became_active":"2024-03-08T08:28:30.880615+0000","last_became_peered":"2024-03-08T08:28:30.880615+0000","last_unstale":"2024-03-08T08:28:32.886702+0000","last_undegraded":"2024-03-08T08:28:32.886702+0000","last_fullsized":"2024-03-08T08:28:32.886702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:39:07.612726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.886576+0000","last_change":"2024-03-08T08:28:31.896967+0000","last_active":"2024-03-08T08:28:32.886576+0000","last_peered":"2024-03-08T08:28:32.886576+0000","last_clean":"2024-03-08T08:28:32.886576+0000","last_became_active":"2024-03-08T08:28:31.896409+0000","last_became_peered":"2024-03-08T08:28:31.896409+0000","last_unstale":"2024-03-08T08:28:32.886576+0000","last_undegraded":"2024-03-08T08:28:32.886576+0000","last_fullsized":"2024-03-08T08:28:32.886576+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:43:45.842491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883589+0000","last_change":"2024-03-08T08:28:30.873910+0000","last_active":"2024-03-08T08:28:32.883589+0000","last_peered":"2024-03-08T08:28:32.883589+0000","last_clean":"2024-03-08T08:28:32.883589+0000","last_became_active":"2024-03-08T08:28:30.872969+0000","last_became_peered":"2024-03-08T08:28:30.872969+0000","last_unstale":"2024-03-08T08:28:32.883589+0000","last_undegraded":"2024-03-08T08:28:32.883589+0000","last_fullsized":"2024-03-08T08:28:32.883589+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:39:07.612726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735120+0000","last_change":"2024-03-08T08:28:31.886419+0000","last_active":"2024-03-08T08:28:33.735120+0000","last_peered":"2024-03-08T08:28:33.735120+0000","last_clean":"2024-03-08T08:28:33.735120+0000","last_became_active":"2024-03-08T08:28:31.886138+0000","last_became_peered":"2024-03-08T08:28:31.886138+0000","last_unstale":"2024-03-08T08:28:33.735120+0000","last_undegraded":"2024-03-08T08:28:33.735120+0000","last_fullsized":"2024-03-08T08:28:33.735120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:08:07.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.885096+0000","last_change":"2024-03-08T08:28:30.869083+0000","last_active":"2024-03-08T08:28:32.885096+0000","last_peered":"2024-03-08T08:28:32.885096+0000","last_clean":"2024-03-08T08:28:32.885096+0000","last_became_active":"2024-03-08T08:28:30.868810+0000","last_became_peered":"2024-03-08T08:28:30.868810+0000","last_unstale":"2024-03-08T08:28:32.885096+0000","last_undegraded":"2024-03-08T08:28:32.885096+0000","last_fullsized":"2024-03-08T08:28:32.885096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T14:59:27.688079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.885008+0000","last_change":"2024-03-08T08:28:31.882689+0000","last_active":"2024-03-08T08:28:32.885008+0000","last_peered":"2024-03-08T08:28:32.885008+0000","last_clean":"2024-03-08T08:28:32.885008+0000","last_became_active":"2024-03-08T08:28:31.882369+0000","last_became_peered":"2024-03-08T08:28:31.882369+0000","last_unstale":"2024-03-08T08:28:32.885008+0000","last_undegraded":"2024-03-08T08:28:32.885008+0000","last_fullsized":"2024-03-08T08:28:32.885008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:47:15.122270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735220+0000","last_change":"2024-03-08T08:28:30.881900+0000","last_active":"2024-03-08T08:28:33.735220+0000","last_peered":"2024-03-08T08:28:33.735220+0000","last_clean":"2024-03-08T08:28:33.735220+0000","last_became_active":"2024-03-08T08:28:30.881645+0000","last_became_peered":"2024-03-08T08:28:30.881645+0000","last_unstale":"2024-03-08T08:28:33.735220+0000","last_undegraded":"2024-03-08T08:28:33.735220+0000","last_fullsized":"2024-03-08T08:28:33.735220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T08:43:44.838465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.735677+0000","last_change":"2024-03-08T08:28:31.891946+0000","last_active":"2024-03-08T08:28:33.735677+0000","last_peered":"2024-03-08T08:28:33.735677+0000","last_clean":"2024-03-08T08:28:33.735677+0000","last_became_active":"2024-03-08T08:28:31.891621+0000","last_became_peered":"2024-03-08T08:28:31.891621+0000","last_unstale":"2024-03-08T08:28:33.735677+0000","last_undegraded":"2024-03-08T08:28:33.735677+0000","last_fullsized":"2024-03-08T08:28:33.735677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:30.854233+0000","last_clean_scrub_stamp":"2024-03-08T08:28:30.854233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T09:37:57.838372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:32.883634+0000","last_change":"2024-03-08T08:28:30.873407+0000","last_active":"2024-03-08T08:28:32.883634+0000","last_peered":"2024-03-08T08:28:32.883634+0000","last_clean":"2024-03-08T08:28:32.883634+0000","last_became_active":"2024-03-08T08:28:30.871776+0000","last_became_peered":"2024-03-08T08:28:30.871776+0000","last_unstale":"2024-03-08T08:28:32.883634+0000","last_undegraded":"2024-03-08T08:28:32.883634+0000","last_fullsized":"2024-03-08T08:28:32.883634+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:28:29.850207+0000","last_clean_scrub_stamp":"2024-03-08T08:28:29.850207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T11:35:56.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-08T08:28:33.736258+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:28:33.736258+0000","last_peered":"2024-03-08T08:28:33.736258+0000","last_clean":"2024-03-08T08:28:33.736258+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:28:33.736258+0000","last_undegraded":"2024-03-08T08:28:33.736258+0000","last_fullsized":"2024-03-08T08:28:33.736258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:28:39.529 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-08T08:28:39.529 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-08T08:28:39.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.671+0000 7fbe866a4640 1 Processor -- start 2024-03-08T08:28:39.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.672+0000 7fbe866a4640 1 -- start start 2024-03-08T08:28:39.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.672+0000 7fbe866a4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161470 0x7fbe8006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.672+0000 7fbe866a4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe80161850 0x7fbe800d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.672+0000 7fbe866a4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe800d48a0 con 0x7fbe8006d370 2024-03-08T08:28:39.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.672+0000 7fbe866a4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe800d2ba0 con 0x7fbe80161850 2024-03-08T08:28:39.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.672+0000 7fbe866a4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800d4b80 con 0x7fbe80161470 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161470 0x7fbe8006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161470 0x7fbe8006c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:36738/0 (socket says 172.21.15.105:36738) 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7f7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe80161850 0x7fbe800d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7ffff640 1 -- 172.21.15.105:0/3027116538 learned_addr learned my addr 172.21.15.105:0/3027116538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7ffff640 1 -- 172.21.15.105:0/3027116538 >> v2:172.21.15.144:3300/0 conn(0x7fbe80161850 msgr2=0x7fbe800d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7ffff640 1 --2- 172.21.15.105:0/3027116538 >> v2:172.21.15.144:3300/0 conn(0x7fbe80161850 0x7fbe800d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7ffff640 1 -- 172.21.15.105:0/3027116538 >> v1:172.21.15.144:6789/0 conn(0x7fbe8006d370 legacy=0x7fbe8006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7ffff640 1 -- 172.21.15.105:0/3027116538 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8006c170 con 0x7fbe80161470 2024-03-08T08:28:39.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.673+0000 7fbe7f7fe640 1 --2- 172.21.15.105:0/3027116538 >> v2:172.21.15.144:3300/0 conn(0x7fbe80161850 0x7fbe800d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.674+0000 7fbe7ffff640 1 --2- 172.21.15.105:0/3027116538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161470 0x7fbe8006c830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009870 tx=0x7fbe7002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=144dc959ac8556d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.674+0000 7fbe7effd640 1 -- 172.21.15.105:0/3027116538 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70035020 con 0x7fbe80161470 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.674+0000 7fbe7effd640 1 -- 172.21.15.105:0/3027116538 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70031640 con 0x7fbe80161470 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.674+0000 7fbe7effd640 1 -- 172.21.15.105:0/3027116538 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7002d040 con 0x7fbe80161470 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.675+0000 7fbe866a4640 1 -- 172.21.15.105:0/3027116538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161470 msgr2=0x7fbe8006c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.675+0000 7fbe866a4640 1 --2- 172.21.15.105:0/3027116538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161470 0x7fbe8006c830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009870 tx=0x7fbe7002b8f0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.675+0000 7fbe866a4640 1 -- 172.21.15.105:0/3027116538 shutdown_connections 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.675+0000 7fbe866a4640 1 --2- 172.21.15.105:0/3027116538 >> v2:172.21.15.144:3300/0 conn(0x7fbe80161850 0x7fbe800d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.675+0000 7fbe866a4640 1 --2- 172.21.15.105:0/3027116538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161470 0x7fbe8006c830 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.675+0000 7fbe866a4640 1 -- 172.21.15.105:0/3027116538 >> 172.21.15.105:0/3027116538 conn(0x7fbe800d3ce0 msgr2=0x7fbe800d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.675+0000 7fbe866a4640 1 -- 172.21.15.105:0/3027116538 shutdown_connections 2024-03-08T08:28:39.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.675+0000 7fbe866a4640 1 -- 172.21.15.105:0/3027116538 wait complete. 2024-03-08T08:28:39.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.676+0000 7fbe866a4640 1 Processor -- start 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.676+0000 7fbe866a4640 1 -- start start 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe866a4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe8006d370 0x7fbe801f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe866a4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe80161470 0x7fbe80152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe866a4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161850 0x7fbe800cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe866a4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8006cf40 con 0x7fbe80161470 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe866a4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8006bcb0 con 0x7fbe8006d370 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe866a4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801f8060 con 0x7fbe80161850 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe80161470 0x7fbe80152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe80161470 0x7fbe80152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36718/0 (socket says 172.21.15.105:36718) 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe8006d370 0x7fbe801f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7f7fe640 1 -- 172.21.15.105:0/1752794305 learned_addr learned my addr 172.21.15.105:0/1752794305 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe84c41640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161850 0x7fbe800cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7f7fe640 1 -- 172.21.15.105:0/1752794305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161850 msgr2=0x7fbe800cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7f7fe640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161850 0x7fbe800cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7f7fe640 1 -- 172.21.15.105:0/1752794305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe8006d370 msgr2=0x7fbe801f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7f7fe640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe8006d370 0x7fbe801f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.677+0000 7fbe7f7fe640 1 -- 172.21.15.105:0/1752794305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800c8a60 con 0x7fbe80161470 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.678+0000 7fbe7f7fe640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe80161470 0x7fbe80152010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400e990 tx=0x7fbe7400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.678+0000 7fbe7cff9640 1 -- 172.21.15.105:0/1752794305 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74017070 con 0x7fbe80161470 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.678+0000 7fbe7cff9640 1 -- 172.21.15.105:0/1752794305 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7400c9e0 con 0x7fbe80161470 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.678+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe800c8d40 con 0x7fbe80161470 2024-03-08T08:28:39.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.678+0000 7fbe7cff9640 1 -- 172.21.15.105:0/1752794305 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400cdb0 con 0x7fbe80161470 2024-03-08T08:28:39.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.678+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe800c9300 con 0x7fbe80161470 2024-03-08T08:28:39.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.679+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe80075120 con 0x7fbe80161470 2024-03-08T08:28:39.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.679+0000 7fbe7cff9640 1 -- 172.21.15.105:0/1752794305 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe7401a070 con 0x7fbe80161470 2024-03-08T08:28:39.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.680+0000 7fbe7cff9640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:39.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.680+0000 7fbe7ffff640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:39.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.680+0000 7fbe7cff9640 1 -- 172.21.15.105:0/1752794305 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe7408a320 con 0x7fbe80161470 2024-03-08T08:28:39.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.680+0000 7fbe7ffff640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbe70036000 tx=0x7fbe7003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:39.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.684+0000 7fbe7cff9640 1 -- 172.21.15.105:0/1752794305 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe74058be0 con 0x7fbe80161470 2024-03-08T08:28:39.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.927+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbe8015d810 con 0x7fbe80161470 2024-03-08T08:28:39.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.929+0000 7fbe7cff9640 1 -- 172.21.15.105:0/1752794305 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbe7405cc80 con 0x7fbe80161470 2024-03-08T08:28:39.931 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-03-08T08:28:39.931 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDBy+pl4CRKIhAAtr/P93sD/1absmF733EcnQ== 2024-03-08T08:28:39.931 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2024-03-08T08:28:39.931 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2024-03-08T08:28:39.931 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-08T08:28:39.931 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbe6c06b4d0 msgr2=0x7fbe6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbe70036000 tx=0x7fbe7003c040 comp rx=0 tx=0).stop 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe80161470 msgr2=0x7fbe80152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe80161470 0x7fbe80152010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400e990 tx=0x7fbe7400ee50 comp rx=0 tx=0).stop 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 shutdown_connections 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80161850 0x7fbe800cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe80161470 0x7fbe80152010 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 --2- 172.21.15.105:0/1752794305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe8006d370 0x7fbe801f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 >> 172.21.15.105:0/1752794305 conn(0x7fbe800d3ce0 msgr2=0x7fbe800b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 shutdown_connections 2024-03-08T08:28:39.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:39.931+0000 7fbe866a4640 1 -- 172.21.15.105:0/1752794305 wait complete. 2024-03-08T08:28:39.944 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:28:40.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.042+0000 7f3f15b9a640 1 Processor -- start 2024-03-08T08:28:40.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.042+0000 7f3f15b9a640 1 -- start start 2024-03-08T08:28:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.042+0000 7f3f15b9a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f10161470 0x7f3f1006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f15b9a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f100d48a0 con 0x7f3f1006bcb0 2024-03-08T08:28:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f15b9a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f100d4b80 con 0x7f3f10161850 2024-03-08T08:28:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f15b9a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1006c170 con 0x7f3f10161470 2024-03-08T08:28:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3f1006bcb0 0x7f3f100b2450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38098/0 (socket says 172.21.15.105:38098) 2024-03-08T08:28:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0ffff640 1 -- 172.21.15.105:0/3851160534 learned_addr learned my addr 172.21.15.105:0/3851160534 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762491434 0 0) 0x7f3f100d48a0 con 0x7f3f1006bcb0 2024-03-08T08:28:40.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3efc003520 con 0x7f3f1006bcb0 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0f7fe640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f10161470 0x7f3f1006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3978084015 0 0) 0x7f3efc003520 con 0x7f3f1006bcb0 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f10161470 msgr2=0x7f3f1006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f10161470 0x7f3f1006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 >> v1:172.21.15.144:6789/0 conn(0x7f3f10161850 legacy=0x7f3f1006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1006c450 con 0x7f3f1006bcb0 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f3f08001a10 con 0x7f3f1006bcb0 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 >> v1:172.21.15.105:6789/0 conn(0x7f3f1006bcb0 legacy=0x7f3f100b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f10161470 0x7f3efc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3efc003f60 0x7f3efc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3efc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1006c170 con 0x7f3efc004d40 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc00bb50 con 0x7f3efc003f60 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc00bd50 con 0x7f3f10161470 2024-03-08T08:28:40.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0ffff640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3efc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.043+0000 7f3f0f7fe640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f10161470 0x7f3efc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f07fff640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3efc003f60 0x7f3efc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0ffff640 1 -- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f10161470 msgr2=0x7f3efc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0ffff640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f10161470 0x7f3efc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0ffff640 1 -- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3efc003f60 msgr2=0x7f3efc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0ffff640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3efc003f60 0x7f3efc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0ffff640 1 -- 172.21.15.105:0/3851160534 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc003520 con 0x7f3efc004d40 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0ffff640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3efc0051c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3f1008c5f0 tx=0x7f3efc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad67d932c42c5f28 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800a4a0 con 0x7f3efc004d40 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08034630 con 0x7f3efc004d40 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.044+0000 7f3f0effd640 1 -- 172.21.15.105:0/3851160534 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0802ce60 con 0x7f3efc004d40 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.045+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3851160534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 msgr2=0x7f3efc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.045+0000 7f3f15b9a640 1 --2- 172.21.15.105:0/3851160534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3efc0051c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3f1008c5f0 tx=0x7f3efc005830 comp rx=0 tx=0).stop 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.045+0000 7f3f0ffff640 1 -- 172.21.15.105:0/3851160534 reap_dead start 2024-03-08T08:28:40.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.045+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3851160534 shutdown_connections 2024-03-08T08:28:40.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.045+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3851160534 >> 172.21.15.105:0/3851160534 conn(0x7f3f100d3ce0 msgr2=0x7f3f100d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.045+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3851160534 shutdown_connections 2024-03-08T08:28:40.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.045+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3851160534 wait complete. 2024-03-08T08:28:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.046+0000 7f3f15b9a640 1 Processor -- start 2024-03-08T08:28:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.046+0000 7f3f15b9a640 1 -- start start 2024-03-08T08:28:40.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f15b9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3f100c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f15b9a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f100c7620 0x7f3f1015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f15b9a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f100cc610 0x7f3f100ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f15b9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1006d4f0 con 0x7f3efc004d40 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f15b9a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1006d370 con 0x7f3f100cc610 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f15b9a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1006d670 con 0x7f3f100c7620 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3f100c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3f100c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36740/0 (socket says 172.21.15.105:36740) 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0f7fe640 1 -- 172.21.15.105:0/3450266576 learned_addr learned my addr 172.21.15.105:0/3450266576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f07fff640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f100c7620 0x7f3f1015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0ffff640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f100cc610 0x7f3f100ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0f7fe640 1 -- 172.21.15.105:0/3450266576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f100c7620 msgr2=0x7f3f1015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0f7fe640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f100c7620 0x7f3f1015d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0f7fe640 1 -- 172.21.15.105:0/3450266576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f100cc610 msgr2=0x7f3f100ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0f7fe640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f100cc610 0x7f3f100ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.047+0000 7f3f0f7fe640 1 -- 172.21.15.105:0/3450266576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f100c7ab0 con 0x7f3efc004d40 2024-03-08T08:28:40.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.048+0000 7f3f0f7fe640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3f100c9af0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3f000119a0 tx=0x7f3f00011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.048+0000 7f3f0cff9640 1 -- 172.21.15.105:0/3450266576 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000faf0 con 0x7f3efc004d40 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.048+0000 7f3f0cff9640 1 -- 172.21.15.105:0/3450266576 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0000fc90 con 0x7f3efc004d40 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.048+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f100c7d90 con 0x7f3efc004d40 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.048+0000 7f3f0cff9640 1 -- 172.21.15.105:0/3450266576 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00014440 con 0x7f3efc004d40 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.048+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f100c8350 con 0x7f3efc004d40 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.049+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10075120 con 0x7f3efc004d40 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.049+0000 7f3f0cff9640 1 -- 172.21.15.105:0/3450266576 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f00012040 con 0x7f3efc004d40 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.049+0000 7f3f0cff9640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3ef006b4d0 0x7f3ef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.050+0000 7f3f07fff640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3ef006b4d0 0x7f3ef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.050+0000 7f3f0cff9640 1 -- 172.21.15.105:0/3450266576 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f0008a7a0 con 0x7f3efc004d40 2024-03-08T08:28:40.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.050+0000 7f3f07fff640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3ef006b4d0 0x7f3ef006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f100b7540 tx=0x7f3ef8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.054+0000 7f3f0cff9640 1 -- 172.21.15.105:0/3450266576 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f00019030 con 0x7f3efc004d40 2024-03-08T08:28:40.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.262+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f1006b8e0 con 0x7f3efc004d40 2024-03-08T08:28:40.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.262+0000 7f3f0cff9640 1 -- 172.21.15.105:0/3450266576 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f3f00058fd0 con 0x7f3efc004d40 2024-03-08T08:28:40.264 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:28:40.264 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2759286099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2759286099},{"type":"v1","addr":"172.21.15.144:6834","nonce":2759286099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2788852991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2788852991},{"type":"v1","addr":"172.21.15.144:6835","nonce":2788852991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2226664928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2226664928},{"type":"v1","addr":"172.21.15.105:6835","nonce":2226664928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:28:31.203118+0000","modified":"2024-03-08T08:28:33.213155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2431299564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2431299564},{"type":"v1","addr":"172.21.15.105:6837","nonce":2431299564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T08:28:40.265 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 13 2024-03-08T08:28:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3ef006b4d0 msgr2=0x7f3ef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3ef006b4d0 0x7f3ef006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f100b7540 tx=0x7f3ef8009040 comp rx=0 tx=0).stop 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 msgr2=0x7f3f100c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3f100c9af0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3f000119a0 tx=0x7f3f00011e60 comp rx=0 tx=0).stop 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 shutdown_connections 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3ef006b4d0 0x7f3ef006d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f100cc610 0x7f3f100ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f100c7620 0x7f3f1015d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 --2- 172.21.15.105:0/3450266576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3efc004d40 0x7f3f100c9af0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 >> 172.21.15.105:0/3450266576 conn(0x7f3f100d3ce0 msgr2=0x7f3f1015f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 shutdown_connections 2024-03-08T08:28:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.264+0000 7f3f15b9a640 1 -- 172.21.15.105:0/3450266576 wait complete. 2024-03-08T08:28:40.679 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-08T08:28:31.203118+0000', 'modified': '2024-03-08T08:28:33.213155+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5100}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5100': {'gid': 5100, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6837/2431299564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 2431299564}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 2431299564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T08:28:40.679 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-08T08:28:40.680 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-08T08:28:40.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.779+0000 7f8bcbb0d640 1 Processor -- start 2024-03-08T08:28:40.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.779+0000 7f8bcbb0d640 1 -- start start 2024-03-08T08:28:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bcbb0d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8bc40b4000 0x7f8bc415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bcbb0d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc40cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bcbb0d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc415b700 con 0x7f8bc40cf290 2024-03-08T08:28:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bcbb0d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40cd9e0 con 0x7f8bc40b4000 2024-03-08T08:28:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bcbb0d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc4072120 con 0x7f8bc406cf90 2024-03-08T08:28:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bc98a9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8bc40b4000 0x7f8bc415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bc98a9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8bc40b4000 0x7f8bc415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36746/0 (socket says 172.21.15.105:36746) 2024-03-08T08:28:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bc98a9640 1 -- 172.21.15.105:0/1799721169 learned_addr learned my addr 172.21.15.105:0/1799721169 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bc98a9640 1 -- 172.21.15.105:0/1799721169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 msgr2=0x7f8bc40cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/1799721169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc40cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/1799721169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc40cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bc98a9640 1 -- 172.21.15.105:0/1799721169 >> v1:172.21.15.144:6789/0 conn(0x7f8bc40cf290 legacy=0x7f8bc40b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bc98a9640 1 -- 172.21.15.105:0/1799721169 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4072400 con 0x7f8bc40b4000 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.780+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/1799721169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc40cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.781+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/1799721169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40b4000 0x7f8bc415b1c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000bb00 tx=0x7f8bc002dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8630f135d8d8e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.781+0000 7f8bc88a7640 1 -- 172.21.15.105:0/1799721169 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0034070 con 0x7f8bc40b4000 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.781+0000 7f8bc88a7640 1 -- 172.21.15.105:0/1799721169 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc002ec50 con 0x7f8bc40b4000 2024-03-08T08:28:40.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.781+0000 7f8bc88a7640 1 -- 172.21.15.105:0/1799721169 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0038400 con 0x7f8bc40b4000 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.782+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/1799721169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40b4000 msgr2=0x7f8bc415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.782+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/1799721169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40b4000 0x7f8bc415b1c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000bb00 tx=0x7f8bc002dea0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.782+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/1799721169 shutdown_connections 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.782+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/1799721169 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc40cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.782+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/1799721169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40b4000 0x7f8bc415b1c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.782+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/1799721169 >> 172.21.15.105:0/1799721169 conn(0x7f8bc40b1740 msgr2=0x7f8bc40b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.782+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/1799721169 shutdown_connections 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.782+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/1799721169 wait complete. 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.783+0000 7f8bcbb0d640 1 Processor -- start 2024-03-08T08:28:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.783+0000 7f8bcbb0d640 1 -- start start 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bcbb0d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc41539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bcbb0d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b4000 0x7f8bc40cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bcbb0d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40cf290 0x7f8bc4157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bcbb0d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415a500 con 0x7f8bc40cf290 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bcbb0d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40726e0 con 0x7f8bc40b4000 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bcbb0d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc406d420 con 0x7f8bc406cf90 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40cf290 0x7f8bc4157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40cf290 0x7f8bc4157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36758/0 (socket says 172.21.15.105:36758) 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 -- 172.21.15.105:0/493566722 learned_addr learned my addr 172.21.15.105:0/493566722 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc41539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b4000 0x7f8bc40cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 -- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 msgr2=0x7f8bc41539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc41539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 -- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b4000 msgr2=0x7f8bc40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b4000 0x7f8bc40cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 -- 172.21.15.105:0/493566722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc413f2c0 con 0x7f8bc40cf290 2024-03-08T08:28:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b4000 0x7f8bc40cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40cf290 0x7f8bc4157830 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00cef0 tx=0x7f8bbc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bba7fc640 1 -- 172.21.15.105:0/493566722 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc01a070 con 0x7f8bc40cf290 2024-03-08T08:28:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bba7fc640 1 -- 172.21.15.105:0/493566722 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc00d860 con 0x7f8bc40cf290 2024-03-08T08:28:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bba7fc640 1 -- 172.21.15.105:0/493566722 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00dc50 con 0x7f8bc40cf290 2024-03-08T08:28:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/493566722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc413f530 con 0x7f8bc40cf290 2024-03-08T08:28:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.784+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/493566722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc413faf0 con 0x7f8bc40cf290 2024-03-08T08:28:40.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.786+0000 7f8bba7fc640 1 -- 172.21.15.105:0/493566722 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bbc01d070 con 0x7f8bc40cf290 2024-03-08T08:28:40.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.786+0000 7f8bba7fc640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8ba806b4d0 0x7f8ba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.786+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8ba806b4d0 0x7f8ba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.786+0000 7f8bba7fc640 1 -- 172.21.15.105:0/493566722 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bbc08aa30 con 0x7f8bc40cf290 2024-03-08T08:28:40.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.787+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8ba806b4d0 0x7f8ba806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0039000 tx=0x7f8bc0034000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.787+0000 7f8bce7bf740 1 Processor -- start 2024-03-08T08:28:40.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.787+0000 7f8bce7bf740 1 -- start start 2024-03-08T08:28:40.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c216ea870 0x564c216eac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c216ee8c0 0x564c216f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c2171a600 0x564c2171aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c216546b0 con 0x564c216ee8c0 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c215fda60 con 0x564c216ea870 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c21650750 con 0x564c2171a600 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bc90a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c216ee8c0 0x564c216f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bc90a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c216ee8c0 0x564c216f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36764/0 (socket says 172.21.15.105:36764) 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bc90a8640 1 -- 172.21.15.105:0/1346055525 learned_addr learned my addr 172.21.15.105:0/1346055525 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c2171a600 0x564c2171aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.788+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c216ea870 0x564c216eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bc90a8640 1 -- 172.21.15.105:0/1346055525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c2171a600 msgr2=0x564c2171aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c2171a600 0x564c2171aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bc90a8640 1 -- 172.21.15.105:0/1346055525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c216ea870 msgr2=0x564c216eac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c216ea870 0x564c216eac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bc90a8640 1 -- 172.21.15.105:0/1346055525 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2161bf30 con 0x564c216ee8c0 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c216ee8c0 0x564c216f0360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400b820 tx=0x7f8bb400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd262fe3b78e531e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8ba77fe640 1 -- 172.21.15.105:0/1346055525 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4004020 con 0x564c216ee8c0 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8ba77fe640 1 -- 172.21.15.105:0/1346055525 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40041c0 con 0x564c216ee8c0 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8ba77fe640 1 -- 172.21.15.105:0/1346055525 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4004390 con 0x564c216ee8c0 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bce7bf740 1 -- 172.21.15.105:0/1346055525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c216ee8c0 msgr2=0x564c216f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c216ee8c0 0x564c216f0360 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400b820 tx=0x7f8bb400bce0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bce7bf740 1 -- 172.21.15.105:0/1346055525 shutdown_connections 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c2171a600 0x564c2171aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c216ee8c0 0x564c216f0360 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/1346055525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c216ea870 0x564c216eac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.789+0000 7f8bce7bf740 1 -- 172.21.15.105:0/1346055525 >> 172.21.15.105:0/1346055525 conn(0x564c216ec8f0 msgr2=0x564c216ee1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.790+0000 7f8bce7bf740 1 -- 172.21.15.105:0/1346055525 shutdown_connections 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.790+0000 7f8bce7bf740 1 -- 172.21.15.105:0/1346055525 wait complete. 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.790+0000 7f8bce7bf740 1 -- start start 2024-03-08T08:28:40.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.790+0000 7f8bce7bf740 1 Processor -- start 2024-03-08T08:28:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c21734dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c2171ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 0x564c2171cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c216eed50 con 0x564c2171a600 2024-03-08T08:28:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c21610ea0 con 0x564c21715720 2024-03-08T08:28:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c216ef030 con 0x564c217322d0 2024-03-08T08:28:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bce7bf740 10 client.0 _ll_register_callbacks cb 0x564c21610870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:28:40.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc98a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c21734dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc98a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c21734dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36778/0 (socket says 172.21.15.105:36778) 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc98a9640 1 -- 172.21.15.105:0/4041570661 learned_addr learned my addr 172.21.15.105:0/4041570661 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c2171ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 0x564c2171cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc98a9640 1 -- 172.21.15.105:0/4041570661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 msgr2=0x564c2171cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 0x564c2171cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc98a9640 1 -- 172.21.15.105:0/4041570661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 msgr2=0x564c2171ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c2171ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.791+0000 7f8bc98a9640 1 -- 172.21.15.105:0/4041570661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c21611270 con 0x564c2171a600 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c21734dc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8bc002e4b0 tx=0x7f8bc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cdbc37dccbcb8e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8ba4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4041570661 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc002ec60 con 0x564c2171a600 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4041570661 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bc0038950 con 0x564c2171a600 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 1 -- 172.21.15.105:0/4041570661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bb40041c0 con 0x564c2171a600 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4041570661 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc005f050 con 0x564c2171a600 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 10 client.5301 fetch_fsmap learned FSMap version 13 2024-03-08T08:28:40.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 1 -- 172.21.15.105:0/4041570661 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x564c213692d0 con 0x564c2171a600 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4041570661 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40041c0 con 0x564c2171a600 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4041570661 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8bc0062020 con 0x564c2171a600 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 10 client.5301 fetch_fsmap finished waiting for FSMap version 13 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 10 client.5301 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 20 client.5301 populate_metadata read hostname 'smithi105' 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x564c21749900 0x564c21754500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 4 client.5301 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bce7bf740 1 -- 172.21.15.105:0/4041570661 --> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c21661bf0 con 0x564c21749900 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.792+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x564c21749900 0x564c21754500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8ba4ff9640 10 client.5301 ms_handle_connect on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x564c21749900 0x564c21754500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=337b78d0908455e0 server_cookie=b133422373ac0bb7 in_seq=0 out_seq=0 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/4041570661 <== mds.0 v2:172.21.15.105:6836/2431299564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c21661bf0 con 0x564c21749900 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8ba4ff9640 10 client.5301 handle_command_reply: tid=0 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 client.5301 shutdown 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 -- 172.21.15.105:0/4041570661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 msgr2=0x564c21734dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c21734dc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8bc002e4b0 tx=0x7f8bc0004730 comp rx=0 tx=0).stop 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 -- 172.21.15.105:0/4041570661 shutdown_connections 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x564c21749900 0x564c21754500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 0x564c2171cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c2171ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8ba4ff9640 0 client.5301 ms_handle_reset on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/4041570661 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c21734dc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 -- 172.21.15.105:0/4041570661 >> 172.21.15.105:0/4041570661 conn(0x564c216ea870 msgr2=0x564c216e9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.793+0000 7f8bce7bf740 1 -- 172.21.15.105:0/4041570661 shutdown_connections 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.794+0000 7f8bce7bf740 1 -- 172.21.15.105:0/4041570661 wait complete. 2024-03-08T08:28:40.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.794+0000 7f8bce7bf740 20 client.5301 trim_cache size 0 max 16384 2024-03-08T08:28:40.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.809+0000 7f8bce7bf740 1 Processor -- start 2024-03-08T08:28:40.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.809+0000 7f8bce7bf740 1 -- start start 2024-03-08T08:28:40.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c21715720 0x564c216e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c2171a600 0x564c217195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c217322d0 0x564c21719b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41fada0 con 0x564c217322d0 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc4141ff0 con 0x564c2171a600 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41fa7a0 con 0x564c21715720 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c217322d0 0x564c21719b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c217322d0 0x564c21719b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36786/0 (socket says 172.21.15.105:36786) 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 -- 172.21.15.105:0/883978464 learned_addr learned my addr 172.21.15.105:0/883978464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 -- 172.21.15.105:0/883978464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c21715720 msgr2=0x564c216e9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/883978464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c21715720 0x564c216e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 -- 172.21.15.105:0/883978464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c2171a600 msgr2=0x564c217195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/883978464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c2171a600 0x564c217195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 -- 172.21.15.105:0/883978464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c214336b0 con 0x564c217322d0 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/883978464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c217322d0 0x564c21719b10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc088f50 tx=0x7f8bbc021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ec91341ab1f955 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8ba6ffd640 1 -- 172.21.15.105:0/883978464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc057610 con 0x564c217322d0 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8ba6ffd640 1 -- 172.21.15.105:0/883978464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc05d0b0 con 0x564c217322d0 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.810+0000 7f8ba6ffd640 1 -- 172.21.15.105:0/883978464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0614e0 con 0x564c217322d0 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 -- 172.21.15.105:0/883978464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c217322d0 msgr2=0x564c21719b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/883978464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c217322d0 0x564c21719b10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc088f50 tx=0x7f8bbc021040 comp rx=0 tx=0).stop 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 -- 172.21.15.105:0/883978464 shutdown_connections 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/883978464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c217322d0 0x564c21719b10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/883978464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c2171a600 0x564c217195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/883978464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c21715720 0x564c216e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 -- 172.21.15.105:0/883978464 >> 172.21.15.105:0/883978464 conn(0x564c216ea870 msgr2=0x564c216e5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 -- 172.21.15.105:0/883978464 shutdown_connections 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.811+0000 7f8bce7bf740 1 -- 172.21.15.105:0/883978464 wait complete. 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.812+0000 7f8bce7bf740 1 -- start start 2024-03-08T08:28:40.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.812+0000 7f8bce7bf740 1 Processor -- start 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.812+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c216e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.812+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c2171d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.812+0000 7f8bce7bf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 0x564c21735810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.812+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c217353c0 con 0x564c2171a600 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2171bf70 con 0x564c21715720 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bce7bf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c2171e200 con 0x564c217322d0 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bce7bf740 10 client.0 _ll_register_callbacks cb 0x7f8bc0045050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c2171d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c2171d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36794/0 (socket says 172.21.15.105:36794) 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 -- 172.21.15.105:0/3849433247 learned_addr learned my addr 172.21.15.105:0/3849433247 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c216e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bca0aa640 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 0x564c21735810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 -- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 msgr2=0x564c21735810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 0x564c21735810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 -- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 msgr2=0x564c216e9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c216e9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 -- 172.21.15.105:0/3849433247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c2171a360 con 0x564c2171a600 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c216e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bc90a8640 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c2171d2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x564c2171ecf0 tx=0x7f8bb4004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27878ee14b4d1be2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8ba4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/3849433247 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4027b80 con 0x564c2171a600 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/3849433247 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bb4027d20 con 0x564c2171a600 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bce7bf740 1 -- 172.21.15.105:0/3849433247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bbc05d0b0 con 0x564c2171a600 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/3849433247 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bb4034050 con 0x564c2171a600 2024-03-08T08:28:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bce7bf740 10 client.5307 fetch_fsmap learned FSMap version 13 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8bce7bf740 1 -- 172.21.15.105:0/3849433247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x564c2171d830 con 0x564c2171a600 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/3849433247 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc05d0b0 con 0x564c2171a600 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.813+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/3849433247 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8bb4037020 con 0x564c2171a600 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 10 client.5307 fetch_fsmap finished waiting for FSMap version 13 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 10 client.5307 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 20 client.5307 populate_metadata read hostname 'smithi105' 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x564c21713780 0x564c21713b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 4 client.5307 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 -- 172.21.15.105:0/3849433247 --> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564c216eb630 con 0x564c21713780 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x564c21713780 0x564c21713b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8ba4ff9640 10 client.5307 ms_handle_connect on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bc98a9640 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x564c21713780 0x564c21713b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90ed5721586da8f server_cookie=506d5ab7350cce2 in_seq=0 out_seq=0 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8ba4ff9640 1 -- 172.21.15.105:0/3849433247 <== mds.0 v2:172.21.15.105:6836/2431299564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564c216eb630 con 0x564c21713780 2024-03-08T08:28:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8ba4ff9640 10 client.5307 handle_command_reply: tid=0 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 client.5307 shutdown 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 -- 172.21.15.105:0/3849433247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 msgr2=0x564c2171d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c2171d2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x564c2171ecf0 tx=0x7f8bb4004680 comp rx=0 tx=0).stop 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 -- 172.21.15.105:0/3849433247 shutdown_connections 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x564c21713780 0x564c21713b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564c217322d0 0x564c21735810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8ba4ff9640 0 client.5307 ms_handle_reset on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c2171a600 0x564c2171d2f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 --2- 172.21.15.105:0/3849433247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c21715720 0x564c216e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 -- 172.21.15.105:0/3849433247 >> 172.21.15.105:0/3849433247 conn(0x564c216ea870 msgr2=0x564c21733d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 -- 172.21.15.105:0/3849433247 shutdown_connections 2024-03-08T08:28:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.814+0000 7f8bce7bf740 1 -- 172.21.15.105:0/3849433247 wait complete. 2024-03-08T08:28:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bce7bf740 20 client.5307 trim_cache size 0 max 16384 2024-03-08T08:28:40.828 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:28:40.828 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "1afff3d7-46c3-4cd2-9212-c89ee069e495", 2024-03-08T08:28:40.828 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-03-08T08:28:40.828 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5100, 2024-03-08T08:28:40.828 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-08T08:28:40.828 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-08T08:28:40.828 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-08T08:28:40.829 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 8.9380746609999999, 2024-03-08T08:28:40.829 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 13, 2024-03-08T08:28:40.829 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 33, 2024-03-08T08:28:40.829 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 33, 2024-03-08T08:28:40.829 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 15.835953288000001 2024-03-08T08:28:40.829 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:28:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8ba806b4d0 msgr2=0x7f8ba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8ba806b4d0 0x7f8ba806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0039000 tx=0x7f8bc0034000 comp rx=0 tx=0).stop 2024-03-08T08:28:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40cf290 msgr2=0x7f8bc4157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40cf290 0x7f8bc4157830 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00cef0 tx=0x7f8bbc00a710 comp rx=0 tx=0).stop 2024-03-08T08:28:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/493566722 shutdown_connections 2024-03-08T08:28:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8ba806b4d0 0x7f8ba806d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bc40cf290 0x7f8bc4157830 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40b4000 0x7f8bc40cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 --2- 172.21.15.105:0/493566722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc406cf90 0x7f8bc41539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/493566722 >> 172.21.15.105:0/493566722 conn(0x7f8bc40b1740 msgr2=0x7f8bc40b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/493566722 shutdown_connections 2024-03-08T08:28:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.815+0000 7f8bcbb0d640 1 -- 172.21.15.105:0/493566722 wait complete. 2024-03-08T08:28:40.832 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T08:28:40.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.931+0000 7f3db3655640 1 Processor -- start 2024-03-08T08:28:40.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.932+0000 7f3db3655640 1 -- start start 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.932+0000 7f3db3655640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3dac06bcb0 0x7f3dac06c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.932+0000 7f3db3655640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dac0d48a0 con 0x7f3dac161470 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.932+0000 7f3db3655640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dac0d4b80 con 0x7f3dac161850 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.932+0000 7f3db3655640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dac06c170 con 0x7f3dac06bcb0 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.933+0000 7f3db0bf0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3dac06bcb0 0x7f3dac06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.933+0000 7f3db13f1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3dac161470 0x7f3dac06b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:38100/0 (socket says 172.21.15.105:38100) 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.933+0000 7f3db13f1640 1 -- 172.21.15.105:0/3789867524 learned_addr learned my addr 172.21.15.105:0/3789867524 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.933+0000 7f3db0bf0640 1 -- 172.21.15.105:0/3789867524 >> v1:172.21.15.144:6789/0 conn(0x7f3dac161850 legacy=0x7f3dac0b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.933+0000 7f3db0bf0640 1 -- 172.21.15.105:0/3789867524 >> v1:172.21.15.105:6789/0 conn(0x7f3dac161470 legacy=0x7f3dac06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.933+0000 7f3db0bf0640 1 -- 172.21.15.105:0/3789867524 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac06c450 con 0x7f3dac06bcb0 2024-03-08T08:28:40.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.933+0000 7f3da3fff640 1 -- 172.21.15.105:0/3789867524 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938235010 0 0) 0x7f3dac0d48a0 con 0x7f3dac161470 2024-03-08T08:28:40.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3db0bf0640 1 --2- 172.21.15.105:0/3789867524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac06bcb0 0x7f3dac06c830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097a0 tx=0x7f3da802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998cd647f45f31dc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3da3fff640 1 -- 172.21.15.105:0/3789867524 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da80315a0 con 0x7f3dac06bcb0 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3da3fff640 1 -- 172.21.15.105:0/3789867524 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8037070 con 0x7f3dac06bcb0 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3da3fff640 1 -- 172.21.15.105:0/3789867524 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8031960 con 0x7f3dac06bcb0 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3db3655640 1 -- 172.21.15.105:0/3789867524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac06bcb0 msgr2=0x7f3dac06c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3db3655640 1 --2- 172.21.15.105:0/3789867524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac06bcb0 0x7f3dac06c830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097a0 tx=0x7f3da802b840 comp rx=0 tx=0).stop 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3db3655640 1 -- 172.21.15.105:0/3789867524 shutdown_connections 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3db3655640 1 --2- 172.21.15.105:0/3789867524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac06bcb0 0x7f3dac06c830 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3db3655640 1 -- 172.21.15.105:0/3789867524 >> 172.21.15.105:0/3789867524 conn(0x7f3dac0d3ce0 msgr2=0x7f3dac0d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3db3655640 1 -- 172.21.15.105:0/3789867524 shutdown_connections 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.934+0000 7f3db3655640 1 -- 172.21.15.105:0/3789867524 wait complete. 2024-03-08T08:28:40.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.935+0000 7f3db3655640 1 Processor -- start 2024-03-08T08:28:40.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db3655640 1 -- start start 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db3655640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dac06bcb0 0x7f3dac15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db3655640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac161470 0x7f3dac159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db3655640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac161850 0x7f3dac0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db3655640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac06d4f0 con 0x7f3dac161470 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db3655640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac06d370 con 0x7f3dac161850 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db3655640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac06d670 con 0x7f3dac06bcb0 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db0bf0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac161470 0x7f3dac159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db0bf0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac161470 0x7f3dac159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36802/0 (socket says 172.21.15.105:36802) 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db0bf0640 1 -- 172.21.15.105:0/1834044371 learned_addr learned my addr 172.21.15.105:0/1834044371 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db13f1640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dac06bcb0 0x7f3dac15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.936+0000 7f3db1bf2640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac161850 0x7f3dac0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db0bf0640 1 -- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dac06bcb0 msgr2=0x7f3dac15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db0bf0640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dac06bcb0 0x7f3dac15b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db0bf0640 1 -- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac161850 msgr2=0x7f3dac0c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db0bf0640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac161850 0x7f3dac0c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db0bf0640 1 -- 172.21.15.105:0/1834044371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac0b6c30 con 0x7f3dac161470 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db13f1640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dac06bcb0 0x7f3dac15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db0bf0640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac161470 0x7f3dac159860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097a0 tx=0x7f3da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3da1ffb640 1 -- 172.21.15.105:0/1834044371 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8004030 con 0x7f3dac161470 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3da1ffb640 1 -- 172.21.15.105:0/1834044371 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da80041f0 con 0x7f3dac161470 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3da1ffb640 1 -- 172.21.15.105:0/1834044371 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8032640 con 0x7f3dac161470 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac0b6f10 con 0x7f3dac161470 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.937+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac0b74d0 con 0x7f3dac161470 2024-03-08T08:28:40.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.938+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dac075120 con 0x7f3dac161470 2024-03-08T08:28:40.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.939+0000 7f3da1ffb640 1 -- 172.21.15.105:0/1834044371 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3da80327e0 con 0x7f3dac161470 2024-03-08T08:28:40.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.939+0000 7f3da1ffb640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3d9006b4d0 0x7f3d9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:40.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.939+0000 7f3da1ffb640 1 -- 172.21.15.105:0/1834044371 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3da802f030 con 0x7f3dac161470 2024-03-08T08:28:40.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.939+0000 7f3db13f1640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3d9006b4d0 0x7f3d9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:40.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.939+0000 7f3db13f1640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3d9006b4d0 0x7f3d9006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c000f60 tx=0x7f3d9c007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:40.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:40.943+0000 7f3da1ffb640 1 -- 172.21.15.105:0/1834044371 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3da8041070 con 0x7f3dac161470 2024-03-08T08:28:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.151+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3dac074600 con 0x7f3dac161470 2024-03-08T08:28:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.151+0000 7f3da1ffb640 1 -- 172.21.15.105:0/1834044371 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3da807b3e0 con 0x7f3dac161470 2024-03-08T08:28:41.154 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3d9006b4d0 msgr2=0x7f3d9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3d9006b4d0 0x7f3d9006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c000f60 tx=0x7f3d9c007c10 comp rx=0 tx=0).stop 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac161470 msgr2=0x7f3dac159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac161470 0x7f3dac159860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097a0 tx=0x7f3da80046c0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 shutdown_connections 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f3d9006b4d0 0x7f3d9006d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac161850 0x7f3dac0c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac161470 0x7f3dac159860 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 --2- 172.21.15.105:0/1834044371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dac06bcb0 0x7f3dac15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 >> 172.21.15.105:0/1834044371 conn(0x7f3dac0d3ce0 msgr2=0x7f3dac0d40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.153+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 shutdown_connections 2024-03-08T08:28:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.154+0000 7f3db3655640 1 -- 172.21.15.105:0/1834044371 wait complete. 2024-03-08T08:28:41.166 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-08T08:28:41.166 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T08:28:41.166 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-03-08T08:28:41.166 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-08T08:28:41.166 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T08:28:41.166 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-08T08:28:41.167 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-08T08:28:41.167 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T08:28:41.167 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T08:28:41.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.167 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-03-08T08:28:41.182 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:28:41.182 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:28:41.182 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:28:41.182 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:41.182 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-08T08:28:41.182 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42661sec preferred_lft 42661sec 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-08T08:28:41.183 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:68:08:f2:11:a9 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::20ed:97ff:fe3a:c855/64 scope link 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:68:08:f2:11:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::68:8ff:fef2:11a9/64 scope link 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T08:28:41.184 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0a:d2:53:3c:79:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T08:28:41.185 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::8d2:53ff:fe3c:79ba/64 scope link 2024-03-08T08:28:41.185 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:41.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.185 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T08:28:41.248 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-03-08T08:28:41.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.250 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-03-08T08:28:41.305 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-03-08T08:28:41.305 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T08:28:41.305 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T08:28:41.305 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T08:28:41.305 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T08:28:41.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.305 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-03-08T08:28:41.305 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-08T08:28:41.305 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T08:28:41.305 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T08:28:41.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T08:28:41.306 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-08T08:28:41.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.399 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-08T08:28:41.413 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-08T08:28:41.414 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T08:28:41.414 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T08:28:41.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.414 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-03-08T08:28:41.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:28:41.481 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-08T08:28:41.535 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-08T08:28:41.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:28:41.536 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-08T08:28:41.591 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-08T08:28:41.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:41.674 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:41.675 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:28:41.683 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:41.683 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:28:41.683 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.144:3300,172.21.15.144:3301" 2024-03-08T08:28:41.683 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 0@1afff3d7-46c3-4cd2-9212-c89ee069e495.cephfs=/ 2024-03-08T08:28:41.683 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.105:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2024-03-08T08:28:41.685 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-08T08:28:41.685 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-03-08T08:28:41.685 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T08:28:41.685 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T08:28:41.685 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T08:28:41.686 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T08:28:41.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-08T08:28:41.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:41.720 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:41.780 INFO:teuthology.orchestra.run.smithi105.stdout:client5089 2024-03-08T08:28:41.780 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:41.781 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:41.836 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-08T08:28:41.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.975+0000 7fd1a6815640 1 Processor -- start 2024-03-08T08:28:41.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.975+0000 7fd1a6815640 1 -- start start 2024-03-08T08:28:41.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.975+0000 7fd1a6815640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1a0072440 0x7fd1a0071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:41.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.975+0000 7fd1a6815640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a00b4d00 0x7fd1a015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.975+0000 7fd1a6815640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1a015b4a0 0x7fd1a006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.975+0000 7fd1a6815640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a015b940 con 0x7fd1a0072440 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.975+0000 7fd1a6815640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a015bc20 con 0x7fd1a015b4a0 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.975+0000 7fd1a6815640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0070840 con 0x7fd1a00b4d00 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1a0072440 0x7fd1a0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1a0072440 0x7fd1a0071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36812/0 (socket says 172.21.15.105:36812) 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19ffff640 1 -- 172.21.15.105:0/3010426850 learned_addr learned my addr 172.21.15.105:0/3010426850 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd1a4db2640 1 --2- 172.21.15.105:0/3010426850 >> v2:172.21.15.144:3300/0 conn(0x7fd1a015b4a0 0x7fd1a006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19f7fe640 1 --2- 172.21.15.105:0/3010426850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a00b4d00 0x7fd1a015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19ffff640 1 -- 172.21.15.105:0/3010426850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a00b4d00 msgr2=0x7fd1a015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19ffff640 1 --2- 172.21.15.105:0/3010426850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a00b4d00 0x7fd1a015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19ffff640 1 -- 172.21.15.105:0/3010426850 >> v2:172.21.15.144:3300/0 conn(0x7fd1a015b4a0 msgr2=0x7fd1a006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19ffff640 1 --2- 172.21.15.105:0/3010426850 >> v2:172.21.15.144:3300/0 conn(0x7fd1a015b4a0 0x7fd1a006e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.976+0000 7fd19ffff640 1 -- 172.21.15.105:0/3010426850 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a0070b20 con 0x7fd1a0072440 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd19ffff640 1 --2- 172.21.15.105:0/3010426850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a0072440 0x7fd1a0071020 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0158aa0 tx=0x7fd19002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffad256d8237f432 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd19effd640 1 -- 172.21.15.105:0/3010426850 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19003b070 con 0x7fd1a0072440 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd19effd640 1 -- 172.21.15.105:0/3010426850 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd190036440 con 0x7fd1a0072440 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd19effd640 1 -- 172.21.15.105:0/3010426850 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190036810 con 0x7fd1a0072440 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 -- 172.21.15.105:0/3010426850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a0072440 msgr2=0x7fd1a0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3010426850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a0072440 0x7fd1a0071020 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0158aa0 tx=0x7fd19002e0e0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 -- 172.21.15.105:0/3010426850 shutdown_connections 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3010426850 >> v2:172.21.15.144:3300/0 conn(0x7fd1a015b4a0 0x7fd1a006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3010426850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a00b4d00 0x7fd1a015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3010426850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a0072440 0x7fd1a0071020 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 -- 172.21.15.105:0/3010426850 >> 172.21.15.105:0/3010426850 conn(0x7fd1a00b1740 msgr2=0x7fd1a00b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 -- 172.21.15.105:0/3010426850 shutdown_connections 2024-03-08T08:28:41.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.977+0000 7fd1a6815640 1 -- 172.21.15.105:0/3010426850 wait complete. 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.979+0000 7fd1a6815640 1 Processor -- start 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.979+0000 7fd1a6815640 1 -- start start 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.979+0000 7fd1a6815640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a0072440 0x7fd1a0152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.979+0000 7fd1a6815640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a00b4d00 0x7fd1a0141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.979+0000 7fd1a6815640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a015b4a0 0x7fd1a015a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.979+0000 7fd1a6815640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00729a0 con 0x7fd1a015b4a0 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.979+0000 7fd1a6815640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0072820 con 0x7fd1a00b4d00 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.979+0000 7fd1a6815640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a006e5e0 con 0x7fd1a0072440 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a015b4a0 0x7fd1a015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a015b4a0 0x7fd1a015a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36816/0 (socket says 172.21.15.105:36816) 2024-03-08T08:28:41.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 -- 172.21.15.105:0/3494321878 learned_addr learned my addr 172.21.15.105:0/3494321878 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd19f7fe640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a00b4d00 0x7fd1a0141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd19ffff640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a0072440 0x7fd1a0152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 -- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a0072440 msgr2=0x7fd1a0152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a0072440 0x7fd1a0152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 -- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a00b4d00 msgr2=0x7fd1a0141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a00b4d00 0x7fd1a0141390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 -- 172.21.15.105:0/3494321878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a0152200 con 0x7fd1a015b4a0 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd19ffff640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a0072440 0x7fd1a0152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd19f7fe640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a00b4d00 0x7fd1a0141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a4db2640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a015b4a0 0x7fd1a015a260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd198007d80 tx=0x7fd19800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd19cff9640 1 -- 172.21.15.105:0/3494321878 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198014420 con 0x7fd1a015b4a0 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd19cff9640 1 -- 172.21.15.105:0/3494321878 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198019070 con 0x7fd1a015b4a0 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a01524e0 con 0x7fd1a015b4a0 2024-03-08T08:28:41.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd19cff9640 1 -- 172.21.15.105:0/3494321878 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198014800 con 0x7fd1a015b4a0 2024-03-08T08:28:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.980+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a015a7a0 con 0x7fd1a015b4a0 2024-03-08T08:28:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.981+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a0075120 con 0x7fd1a015b4a0 2024-03-08T08:28:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.982+0000 7fd19cff9640 1 -- 172.21.15.105:0/3494321878 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1980149a0 con 0x7fd1a015b4a0 2024-03-08T08:28:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.982+0000 7fd19cff9640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd18c06b4d0 0x7fd18c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.982+0000 7fd19ffff640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd18c06b4d0 0x7fd18c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.982+0000 7fd19cff9640 1 -- 172.21.15.105:0/3494321878 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd19808a9c0 con 0x7fd1a015b4a0 2024-03-08T08:28:41.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.983+0000 7fd19ffff640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd18c06b4d0 0x7fd18c06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd190038000 tx=0x7fd190032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:41.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:41.987+0000 7fd19cff9640 1 -- 172.21.15.105:0/3494321878 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd19808ce20 con 0x7fd1a015b4a0 2024-03-08T08:28:42.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.193+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1a0070fe0 con 0x7fd1a015b4a0 2024-03-08T08:28:42.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.194+0000 7fd19cff9640 1 -- 172.21.15.105:0/3494321878 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd19805f030 con 0x7fd1a015b4a0 2024-03-08T08:28:42.196 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T08:28:42.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd18c06b4d0 msgr2=0x7fd18c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:42.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd18c06b4d0 0x7fd18c06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd190038000 tx=0x7fd190032000 comp rx=0 tx=0).stop 2024-03-08T08:28:42.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a015b4a0 msgr2=0x7fd1a015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:42.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a015b4a0 0x7fd1a015a260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd198007d80 tx=0x7fd19800e4b0 comp rx=0 tx=0).stop 2024-03-08T08:28:42.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 shutdown_connections 2024-03-08T08:28:42.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd18c06b4d0 0x7fd18c06d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:42.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1a015b4a0 0x7fd1a015a260 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:42.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a00b4d00 0x7fd1a0141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:42.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 --2- 172.21.15.105:0/3494321878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a0072440 0x7fd1a0152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:42.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 >> 172.21.15.105:0/3494321878 conn(0x7fd1a00b1740 msgr2=0x7fd1a00b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:42.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 shutdown_connections 2024-03-08T08:28:42.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:42.196+0000 7fd1a6815640 1 -- 172.21.15.105:0/3494321878 wait complete. 2024-03-08T08:28:42.208 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-08T08:28:42.209 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T08:28:42.209 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-03-08T08:28:42.209 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-08T08:28:42.209 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:42.209 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-08T08:28:42.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-08T08:28:42.209 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T08:28:42.209 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T08:28:42.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.209 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-03-08T08:28:42.228 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T08:28:42.228 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T08:28:42.228 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T08:28:42.228 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:42.228 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-03-08T08:28:42.228 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:42.228 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42596sec preferred_lft 42596sec 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 96:29:85:55:60:af brd ff:ff:ff:ff:ff:ff 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-08T08:28:42.229 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::dc81:beff:feca:aa78/64 scope link 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether d6:2a:4f:6b:63:46 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::d42a:4fff:fe6b:6346/64 scope link 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 96:29:85:55:60:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::9429:85ff:fe55:60af/64 scope link 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-03-08T08:28:42.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.230 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T08:28:42.298 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-03-08T08:28:42.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.300 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-03-08T08:28:42.357 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-03-08T08:28:42.357 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T08:28:42.357 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T08:28:42.357 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T08:28:42.357 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T08:28:42.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.357 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2024-03-08T08:28:42.358 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-03-08T08:28:42.358 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T08:28:42.358 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T08:28:42.358 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T08:28:42.358 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-03-08T08:28:42.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.456 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-03-08T08:28:42.471 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-08T08:28:42.471 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T08:28:42.471 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-08T08:28:42.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.472 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2024-03-08T08:28:42.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:28:42.540 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:42.595 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-08T08:28:42.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T08:28:42.595 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:42.649 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-08T08:28:42.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.650 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:42.734 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:42.734 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:28:42.741 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.144:3300,172.21.15.144:3301" 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@1afff3d7-46c3-4cd2-9212-c89ee069e495.cephfs=/ 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.105:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2024-03-08T08:28:42.742 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-08T08:28:42.744 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-08T08:28:42.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.744 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-08T08:28:42.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:42.811 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:42.870 INFO:teuthology.orchestra.run.smithi144.stdout:client4853 2024-03-08T08:28:42.871 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:42.871 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:52.927 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T08:28:52.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-08T08:28:53.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.039+0000 7f7259c15640 1 Processor -- start 2024-03-08T08:28:53.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.039+0000 7f7259c15640 1 -- start start 2024-03-08T08:28:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f7259c15640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f72540cf290 0x7f72540b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f7259c15640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72540b4920 con 0x7f725406cf90 2024-03-08T08:28:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f7259c15640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7254072120 con 0x7f72540b4000 2024-03-08T08:28:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f7259c15640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72540cd9e0 con 0x7f72540cf290 2024-03-08T08:28:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f7253fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f725406cf90 0x7f72540cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41508/0 (socket says 172.21.15.105:41508) 2024-03-08T08:28:53.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f7253fff640 1 -- 172.21.15.105:0/2344572426 learned_addr learned my addr 172.21.15.105:0/2344572426 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2344572426 >> v2:172.21.15.144:3300/0 conn(0x7f72540cf290 0x7f72540b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617728823 0 0) 0x7f72540b4920 con 0x7f725406cf90 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.040+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f723c003520 con 0x7f725406cf90 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1646267607 0 0) 0x7f723c003520 con 0x7f725406cf90 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 >> v2:172.21.15.144:3300/0 conn(0x7f72540cf290 msgr2=0x7f72540b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 --2- 172.21.15.105:0/2344572426 >> v2:172.21.15.144:3300/0 conn(0x7f72540cf290 0x7f72540b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 >> v1:172.21.15.144:6789/0 conn(0x7f72540b4000 legacy=0x7f725415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254072400 con 0x7f725406cf90 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f724c0019f0 con 0x7f725406cf90 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 >> v1:172.21.15.105:6789/0 conn(0x7f725406cf90 legacy=0x7f72540cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c0039b0 0x7f723c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f723c004800 0x7f723c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f723c0051c0 0x7f723c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540cd9e0 con 0x7f723c004800 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c00c060 con 0x7f723c0051c0 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c00c260 con 0x7f723c0039b0 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f723c004800 0x7f723c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f7253fff640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f723c0051c0 0x7f723c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72537fe640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c0039b0 0x7f723c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f7252ffd640 1 -- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c0039b0 msgr2=0x7f723c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c0039b0 0x7f723c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f7252ffd640 1 -- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f723c0051c0 msgr2=0x7f723c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f723c0051c0 0x7f723c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f7253fff640 1 -- 172.21.15.105:0/2344572426 reap_dead start 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f7252ffd640 1 -- 172.21.15.105:0/2344572426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c003520 con 0x7f723c004800 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.041+0000 7f72537fe640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c0039b0 0x7f723c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.042+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f723c004800 0x7f723c004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7248009ac0 tx=0x7f723c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c3baa64b177b81 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:53.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.042+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248039070 con 0x7f723c004800 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.042+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724802c6b0 con 0x7f723c004800 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.042+0000 7f72527fc640 1 -- 172.21.15.105:0/2344572426 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248009690 con 0x7f723c004800 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.043+0000 7f7259c15640 1 -- 172.21.15.105:0/2344572426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f723c004800 msgr2=0x7f723c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.043+0000 7f7259c15640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f723c004800 0x7f723c004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7248009ac0 tx=0x7f723c005830 comp rx=0 tx=0).stop 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.043+0000 7f7259c15640 1 -- 172.21.15.105:0/2344572426 shutdown_connections 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.043+0000 7f7259c15640 1 --2- 172.21.15.105:0/2344572426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f723c004800 0x7f723c004c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.043+0000 7f7259c15640 1 -- 172.21.15.105:0/2344572426 >> 172.21.15.105:0/2344572426 conn(0x7f72540b1740 msgr2=0x7f72540b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.043+0000 7f7259c15640 1 -- 172.21.15.105:0/2344572426 shutdown_connections 2024-03-08T08:28:53.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.043+0000 7f7259c15640 1 -- 172.21.15.105:0/2344572426 wait complete. 2024-03-08T08:28:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.044+0000 7f7259c15640 1 Processor -- start 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.044+0000 7f7259c15640 1 -- start start 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7259c15640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c004800 0x7f725413ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7259c15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f725413db10 0x7f72541405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7259c15640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72540af7f0 0x7f72540afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7259c15640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725415a500 con 0x7f725413db10 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7259c15640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540726e0 con 0x7f72540af7f0 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7259c15640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725406d400 con 0x7f723c004800 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f725413db10 0x7f72541405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f725413db10 0x7f72541405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49660/0 (socket says 172.21.15.105:49660) 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 -- 172.21.15.105:0/3916341812 learned_addr learned my addr 172.21.15.105:0/3916341812 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7253fff640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72540af7f0 0x7f72540afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f72537fe640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c004800 0x7f725413ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 -- 172.21.15.105:0/3916341812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c004800 msgr2=0x7f725413ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c004800 0x7f725413ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 -- 172.21.15.105:0/3916341812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72540af7f0 msgr2=0x7f72540afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72540af7f0 0x7f72540afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 -- 172.21.15.105:0/3916341812 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725414c620 con 0x7f725413db10 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f7252ffd640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f725413db10 0x7f72541405e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7248009ac0 tx=0x7f7248004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.045+0000 7f723bfff640 1 -- 172.21.15.105:0/3916341812 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248039050 con 0x7f725413db10 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.046+0000 7f723bfff640 1 -- 172.21.15.105:0/3916341812 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7248045070 con 0x7f725413db10 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.046+0000 7f723bfff640 1 -- 172.21.15.105:0/3916341812 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72480044e0 con 0x7f725413db10 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.046+0000 7f7259c15640 1 -- 172.21.15.105:0/3916341812 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725414c8c0 con 0x7f725413db10 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.046+0000 7f7259c15640 1 -- 172.21.15.105:0/3916341812 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725414ce80 con 0x7f725413db10 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.047+0000 7f723bfff640 1 -- 172.21.15.105:0/3916341812 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7248003a50 con 0x7f725413db10 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.047+0000 7f723bfff640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f724006b500 0x7f724006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.047+0000 7f723bfff640 1 -- 172.21.15.105:0/3916341812 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72480abca0 con 0x7f725413db10 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.047+0000 7f72537fe640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f724006b500 0x7f724006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.048+0000 7f72537fe640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f724006b500 0x7f724006d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7244000c00 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.048+0000 7f725c7fc740 1 Processor -- start 2024-03-08T08:28:53.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.048+0000 7f725c7fc740 1 -- start start 2024-03-08T08:28:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06668a0 0x55ead0666c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead066a8f0 0x55ead066c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0696630 0x55ead0696ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f725c7fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ead05d0e30 con 0x55ead0696630 2024-03-08T08:28:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f725c7fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ead0579d20 con 0x55ead06668a0 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f725c7fc740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ead05cced0 con 0x55ead066a8f0 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f7253fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0696630 0x55ead0696ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f7253fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0696630 0x55ead0696ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49668/0 (socket says 172.21.15.105:49668) 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.049+0000 7f7253fff640 1 -- 172.21.15.105:0/3417594946 learned_addr learned my addr 172.21.15.105:0/3417594946 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f72537fe640 1 --2- 172.21.15.105:0/3417594946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06668a0 0x55ead0666c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f7253fff640 1 -- 172.21.15.105:0/3417594946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead066a8f0 msgr2=0x55ead066c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f7253fff640 1 --2- 172.21.15.105:0/3417594946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead066a8f0 0x55ead066c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f7253fff640 1 -- 172.21.15.105:0/3417594946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06668a0 msgr2=0x55ead0666c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f7253fff640 1 --2- 172.21.15.105:0/3417594946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06668a0 0x55ead0666c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f7253fff640 1 -- 172.21.15.105:0/3417594946 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ead02e53f0 con 0x55ead0696630 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f7253fff640 1 --2- 172.21.15.105:0/3417594946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0696630 0x55ead0696ab0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f72540cf940 tx=0x7f724c001ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6e72be26a72402 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f72397fa640 1 -- 172.21.15.105:0/3417594946 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c019020 con 0x55ead0696630 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f72397fa640 1 -- 172.21.15.105:0/3417594946 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c007980 con 0x55ead0696630 2024-03-08T08:28:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.050+0000 7f72397fa640 1 -- 172.21.15.105:0/3417594946 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c015640 con 0x55ead0696630 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 -- 172.21.15.105:0/3417594946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0696630 msgr2=0x55ead0696ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 --2- 172.21.15.105:0/3417594946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0696630 0x55ead0696ab0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f72540cf940 tx=0x7f724c001ee0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 -- 172.21.15.105:0/3417594946 shutdown_connections 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 --2- 172.21.15.105:0/3417594946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0696630 0x55ead0696ab0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 --2- 172.21.15.105:0/3417594946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead066a8f0 0x55ead066c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 --2- 172.21.15.105:0/3417594946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06668a0 0x55ead0666c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 -- 172.21.15.105:0/3417594946 >> 172.21.15.105:0/3417594946 conn(0x55ead0668920 msgr2=0x55ead066a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 -- 172.21.15.105:0/3417594946 shutdown_connections 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 -- 172.21.15.105:0/3417594946 wait complete. 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 -- start start 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.051+0000 7f725c7fc740 1 Processor -- start 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead0695aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 0x55ead0690190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f725c7fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ead066ad80 con 0x55ead06906d0 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f725c7fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ead056f390 con 0x55ead0696630 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f725c7fc740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ead066b060 con 0x55ead0691300 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f725c7fc740 10 client.0 _ll_register_callbacks cb 0x55ead058cc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f7253fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f7253fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49674/0 (socket says 172.21.15.105:49674) 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f7253fff640 1 -- 172.21.15.105:0/2753129654 learned_addr learned my addr 172.21.15.105:0/2753129654 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.052+0000 7f72537fe640 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead0695aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f7253fff640 1 -- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 msgr2=0x55ead0690190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 0x55ead0690190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f7253fff640 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 0x55ead0690190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f7253fff640 1 -- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 msgr2=0x55ead0695aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f7253fff640 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead0695aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f7253fff640 1 -- 172.21.15.105:0/2753129654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ead03af7d0 con 0x55ead06906d0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f72537fe640 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead0695aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f7253fff640 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06accb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f724c001b60 tx=0x7f724c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a884e426b7510c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f72327fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f72327fc640 1 -- 172.21.15.105:0/2753129654 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c019070 con 0x55ead06906d0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f72327fc640 1 -- 172.21.15.105:0/2753129654 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f724c028640 con 0x55ead06906d0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 1 -- 172.21.15.105:0/2753129654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f724c007980 con 0x55ead06906d0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f72327fc640 1 -- 172.21.15.105:0/2753129654 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f724c0017f0 con 0x55ead06906d0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 10 client.5334 fetch_fsmap learned FSMap version 13 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 1 -- 172.21.15.105:0/2753129654 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55ead0699ff0 con 0x55ead06906d0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f72327fc640 1 -- 172.21.15.105:0/2753129654 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c007980 con 0x55ead06906d0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f72327fc640 1 -- 172.21.15.105:0/2753129654 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f724c026020 con 0x55ead06906d0 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 10 client.5334 fetch_fsmap finished waiting for FSMap version 13 2024-03-08T08:28:53.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 10 client.5334 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 20 client.5334 populate_metadata read hostname 'smithi105' 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x55ead06ad6c0 0x55ead06cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 4 client.5334 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f725c7fc740 1 -- 172.21.15.105:0/2753129654 --> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ead05de370 con 0x55ead06ad6c0 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.053+0000 7f72537fe640 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x55ead06ad6c0 0x55ead06cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f72537fe640 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x55ead06ad6c0 0x55ead06cf440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c523204df5bb535 server_cookie=2b40cfe6ab1fbbcb in_seq=0 out_seq=0 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f72327fc640 10 client.5334 ms_handle_connect on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f72327fc640 1 -- 172.21.15.105:0/2753129654 <== mds.0 v2:172.21.15.105:6836/2431299564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ead05de370 con 0x55ead06ad6c0 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f72327fc640 10 client.5334 handle_command_reply: tid=0 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 client.5334 shutdown 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 -- 172.21.15.105:0/2753129654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 msgr2=0x55ead06accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06accb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f724c001b60 tx=0x7f724c001dc0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 -- 172.21.15.105:0/2753129654 shutdown_connections 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x55ead06ad6c0 0x55ead06cf440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06accb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 0x55ead0690190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2753129654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead0695aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f72327fc640 0 client.5334 ms_handle_reset on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.054+0000 7f725c7fc740 1 -- 172.21.15.105:0/2753129654 >> 172.21.15.105:0/2753129654 conn(0x55ead06668a0 msgr2=0x55ead06655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.055+0000 7f725c7fc740 1 -- 172.21.15.105:0/2753129654 shutdown_connections 2024-03-08T08:28:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.055+0000 7f725c7fc740 1 -- 172.21.15.105:0/2753129654 wait complete. 2024-03-08T08:28:53.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.055+0000 7f725c7fc740 20 client.5334 trim_cache size 0 max 16384 2024-03-08T08:28:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.066+0000 7f725c7fc740 1 Processor -- start 2024-03-08T08:28:53.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f725c7fc740 1 -- start start 2024-03-08T08:28:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06906d0 0x55ead066a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0691300 0x55ead06ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0696630 0x55ead06acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f725c7fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72541fe630 con 0x55ead0691300 2024-03-08T08:28:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f725c7fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72541fd820 con 0x55ead06906d0 2024-03-08T08:28:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f725c7fc740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725415bd80 con 0x55ead0696630 2024-03-08T08:28:53.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f7252ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0691300 0x55ead06ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f7252ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0691300 0x55ead06ac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49690/0 (socket says 172.21.15.105:49690) 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.067+0000 7f7252ffd640 1 -- 172.21.15.105:0/3589366293 learned_addr learned my addr 172.21.15.105:0/3589366293 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7252ffd640 1 -- 172.21.15.105:0/3589366293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0696630 msgr2=0x55ead06acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7252ffd640 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0696630 0x55ead06acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f72537fe640 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06906d0 0x55ead066a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7252ffd640 1 -- 172.21.15.105:0/3589366293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06906d0 msgr2=0x55ead066a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7252ffd640 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06906d0 0x55ead066a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7252ffd640 1 -- 172.21.15.105:0/3589366293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ead0699130 con 0x55ead0691300 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f72537fe640 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06906d0 0x55ead066a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7252ffd640 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0691300 0x55ead06ac750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f724802c320 tx=0x7f7248076800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a814c43bb66de54c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7238ff9640 1 -- 172.21.15.105:0/3589366293 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724807a560 con 0x55ead0691300 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7238ff9640 1 -- 172.21.15.105:0/3589366293 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724803a040 con 0x55ead0691300 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.068+0000 7f7238ff9640 1 -- 172.21.15.105:0/3589366293 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724807e600 con 0x55ead0691300 2024-03-08T08:28:53.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 -- 172.21.15.105:0/3589366293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0691300 msgr2=0x55ead06ac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0691300 0x55ead06ac750 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f724802c320 tx=0x7f7248076800 comp rx=0 tx=0).stop 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 -- 172.21.15.105:0/3589366293 shutdown_connections 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0696630 0x55ead06acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead0691300 0x55ead06ac750 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 --2- 172.21.15.105:0/3589366293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead06906d0 0x55ead066a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 -- 172.21.15.105:0/3589366293 >> 172.21.15.105:0/3589366293 conn(0x55ead06668a0 msgr2=0x55ead068ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 -- 172.21.15.105:0/3589366293 shutdown_connections 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 -- 172.21.15.105:0/3589366293 wait complete. 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 -- start start 2024-03-08T08:28:53.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.069+0000 7f725c7fc740 1 Processor -- start 2024-03-08T08:28:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 0x55ead071bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f725c7fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead071c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f725c7fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725415c380 con 0x55ead06906d0 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f725c7fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ead06cf850 con 0x55ead0696630 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f725c7fc740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ead06ab720 con 0x55ead0691300 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f725c7fc740 10 client.0 _ll_register_callbacks cb 0x7f724c001a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f72537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f72537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49696/0 (socket says 172.21.15.105:49696) 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f72537fe640 1 -- 172.21.15.105:0/2739836423 learned_addr learned my addr 172.21.15.105:0/2739836423 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f7253fff640 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead071c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f72537fe640 1 -- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 msgr2=0x55ead071bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 0x55ead071bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f72537fe640 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 0x55ead071bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f72537fe640 1 -- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 msgr2=0x55ead071c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f72537fe640 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead071c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f72537fe640 1 -- 172.21.15.105:0/2739836423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ead06cc0e0 con 0x55ead06906d0 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.070+0000 7f7253fff640 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead071c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:28:53.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f72537fe640 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06ac850 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55ead0690bb0 tx=0x7f7244001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca88b3b47f87338 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f72327fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f72327fc640 1 -- 172.21.15.105:0/2739836423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724401a040 con 0x55ead06906d0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f72327fc640 1 -- 172.21.15.105:0/2739836423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72440257d0 con 0x55ead06906d0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 1 -- 172.21.15.105:0/2739836423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f724803a040 con 0x55ead06906d0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f72327fc640 1 -- 172.21.15.105:0/2739836423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7244035050 con 0x55ead06906d0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 10 client.5340 fetch_fsmap learned FSMap version 13 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 1 -- 172.21.15.105:0/2739836423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55ead06d1b80 con 0x55ead06906d0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f72327fc640 1 -- 172.21.15.105:0/2739836423 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724803a040 con 0x55ead06906d0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f72327fc640 1 -- 172.21.15.105:0/2739836423 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7244038020 con 0x55ead06906d0 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 10 client.5340 fetch_fsmap finished waiting for FSMap version 13 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 10 client.5340 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 20 client.5340 populate_metadata read hostname 'smithi105' 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x55ead06b5240 0x55ead06941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:53.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 4 client.5340 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f725c7fc740 1 -- 172.21.15.105:0/2739836423 --> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x55ead05d86c0 con 0x55ead06b5240 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.071+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x55ead06b5240 0x55ead06941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f72327fc640 10 client.5340 ms_handle_connect on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f7252ffd640 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x55ead06b5240 0x55ead06941d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da47ad63c686a9 server_cookie=c4caf234bc9e3b06 in_seq=0 out_seq=0 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f72327fc640 1 -- 172.21.15.105:0/2739836423 <== mds.0 v2:172.21.15.105:6836/2431299564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26751 (crc 0 0 0) 0x55ead05d86c0 con 0x55ead06b5240 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f72327fc640 10 client.5340 handle_command_reply: tid=0 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 client.5340 shutdown 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 -- 172.21.15.105:0/2739836423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 msgr2=0x55ead06ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06ac850 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55ead0690bb0 tx=0x7f7244001120 comp rx=0 tx=0).stop 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 -- 172.21.15.105:0/2739836423 shutdown_connections 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x55ead06b5240 0x55ead06941d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ead0696630 0x55ead071c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f72327fc640 0 client.5340 ms_handle_reset on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ead0691300 0x55ead071bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 --2- 172.21.15.105:0/2739836423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ead06906d0 0x55ead06ac850 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 -- 172.21.15.105:0/2739836423 >> 172.21.15.105:0/2739836423 conn(0x55ead06668a0 msgr2=0x55ead066a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 -- 172.21.15.105:0/2739836423 shutdown_connections 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 1 -- 172.21.15.105:0/2739836423 wait complete. 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.072+0000 7f725c7fc740 20 client.5340 trim_cache size 0 max 16384 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker": [ 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "0" 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "1" 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "2" 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": [ 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 64, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 58, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 114163, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 34394, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.012575515, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.001543903, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.015298335, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000344591, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 58, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.090149762, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001554306 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.083 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000220, 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000110 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 112016, 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 12352 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": [ 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.084 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 43, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 41, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 12472, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 127926, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.007239175, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.001941123, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.004658482, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000758973, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 40, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.022399517, 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000559987 2024-03-08T08:28:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 59, 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000007858, 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000133 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 94288 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": [ 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 43, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 36, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 12726, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 24097, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.004345132, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000726580, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.002402733, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.001087782, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 35, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.158673542, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.004533529 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:28:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 16, 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000002258, 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000141 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": [ 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-08T08:28:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": [ 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 40, 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002828475, 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000070711 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": [ 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000010937, 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000005468 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": [ 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 25, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 25, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 25, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003269213, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000130768 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 12, 2024-03-08T08:28:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 10, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 10, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 10, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 1, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 2, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 2, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 49, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 49, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-08T08:28:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-08T08:28:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-08T08:28:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 1 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": [ 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-08T08:28:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_client_metrics": [ 2024-03-08T08:28:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "5100" 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "num_clients": 2 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_client_metrics-cephfs": [ 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "client": "client.4853", 2024-03-08T08:28:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": "0" 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_hits": 23, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_miss": 0, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_read_latency": 0.000000000, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_write_latency": 0.000000000, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_hits": 0, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_miss": 0, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_files": 0, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_inodes": 1, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "pinned_icaps": 1, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "total_inodes": 1, 2024-03-08T08:28:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_ops": 0, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_size": 0, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_ops": 0, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_size": 0 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "client": "client.5089", 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": "0" 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_hits": 5, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_miss": 18, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_read_latency": 0.000000000, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_write_latency": 0.000000000, 2024-03-08T08:28:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_hits": 0, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_miss": 0, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_files": 0, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_inodes": 1, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "pinned_icaps": 1, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "total_inodes": 1, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_ops": 0, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_size": 0, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_ops": 0, 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_size": 0 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": [ 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 6, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 6, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-08T08:28:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4196743, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.004052566, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001350855 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": [ 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 13, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 13, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 12, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 12, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 10, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 10, 2024-03-08T08:28:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 2, 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 2, 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 42112, 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215392 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": [ 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 25, 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-08T08:28:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 25, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 2, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 24, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003011598, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000125483 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-08T08:28:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000257615, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000257615 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": [ 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 24, 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 12, 2024-03-08T08:28:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 7, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": [ 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T08:28:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T08:28:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 225602, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 88, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-08T08:28:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5312, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 75, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 52240, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 213, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-08T08:28:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": [ 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 41, 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 13559, 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 31, 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 41, 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-08T08:28:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 41, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.684986642, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.016706991 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 41, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 65 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 41, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 41, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-08T08:28:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 11, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 3, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 7, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 22, 2024-03-08T08:28:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-08T08:28:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 33, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 1, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 7, 2024-03-08T08:28:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 7, 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 15, 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 1 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": [ 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-08T08:28:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": [ 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": [ 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 144, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 122179, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 144, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 144, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 122179, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": [ 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 41, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 13559, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 41, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 13559, 2024-03-08T08:28:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": [ 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:28:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 41, 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 41, 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 41, 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 41, 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": [ 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x557e3b40e6e0": [ 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 6, 2024-03-08T08:28:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 2439, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 6, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 2439, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-08T08:28:53.135 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 -- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f724006b500 msgr2=0x7f724006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f724006b500 0x7f724006d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7244000c00 tx=0x7f7244009040 comp rx=0 tx=0).stop 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 -- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f725413db10 msgr2=0x7f72541405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f725413db10 0x7f72541405e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7248009ac0 tx=0x7f7248004730 comp rx=0 tx=0).stop 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 -- 172.21.15.105:0/3916341812 shutdown_connections 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f724006b500 0x7f724006d9c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72540af7f0 0x7f72540afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f725413db10 0x7f72541405e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 --2- 172.21.15.105:0/3916341812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f723c004800 0x7f725413ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 -- 172.21.15.105:0/3916341812 >> 172.21.15.105:0/3916341812 conn(0x7f72540b1740 msgr2=0x7f72540b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 -- 172.21.15.105:0/3916341812 shutdown_connections 2024-03-08T08:28:53.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:53.073+0000 7f7259c15640 1 -- 172.21.15.105:0/3916341812 wait complete. 2024-03-08T08:28:53.138 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:53.138 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:53.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:53.154 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:53.209 INFO:teuthology.orchestra.run.smithi105.stdout:client5089 2024-03-08T08:28:53.210 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:53.210 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:53.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:28:53.228 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-08T08:28:53.286 INFO:teuthology.orchestra.run.smithi144.stdout:client4853 2024-03-08T08:28:53.287 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:28:53.287 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:53.302 DEBUG:teuthology.orchestra.run.smithi105:> stdin-killer -- python3 -c ' 2024-03-08T08:28:53.302 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-03-08T08:28:53.302 DEBUG:teuthology.orchestra.run.smithi105:> import uuid 2024-03-08T08:28:53.302 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-08T08:28:53.302 DEBUG:teuthology.orchestra.run.smithi105:> n = 1000 2024-03-08T08:28:53.302 DEBUG:teuthology.orchestra.run.smithi105:> create_hard_links = False 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> if 0 > 0: 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> create_hard_links = True 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> path = "/home/ubuntu/cephtest/mnt.0/test_dir/test_file" 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> dpath = os.path.dirname(path) 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> fnameprefix = os.path.basename(path) 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> os.makedirs(dpath, exist_ok=True) 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> try: 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> for i in range(n): 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> with open(fpath, '"'"'w'"'"') as f: 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> f.write(f"{i}") 2024-03-08T08:28:53.303 DEBUG:teuthology.orchestra.run.smithi105:> if True: 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> f.flush() 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> os.fsync(f.fileno()) 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> if False: 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> os.unlink(fpath) 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> if False: 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> os.fsync(dirfd) 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> if create_hard_links: 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> for j in range(0): 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> if False: 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> os.fsync(dirfd) 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> finally: 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> os.close(dirfd) 2024-03-08T08:28:53.304 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-03-08T08:28:56.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:28:56.915 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T08:28:56.915 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-08T08:28:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.056+0000 7f8a443ac640 1 Processor -- start 2024-03-08T08:28:57.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.056+0000 7f8a443ac640 1 -- start start 2024-03-08T08:28:57.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a443ac640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c1469e0 0x7f8a3c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a443ac640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8a3c0ad870 0x7f8a3c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a443ac640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c06dc30 con 0x7f8a3c06ac80 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a443ac640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3c0adcd0 con 0x7f8a3c0ad870 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a443ac640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c06deb0 con 0x7f8a3c1469e0 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a42949640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8a3c06ac80 0x7f8a3c0b4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53552/0 (socket says 172.21.15.105:53552) 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a42949640 1 -- 172.21.15.105:0/2669202571 learned_addr learned my addr 172.21.15.105:0/2669202571 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a42148640 1 --2- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c1469e0 0x7f8a3c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a41947640 1 --2- 172.21.15.105:0/2669202571 >> v2:172.21.15.144:3300/0 conn(0x7f8a3c0ad870 0x7f8a3c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a41146640 1 -- 172.21.15.105:0/2669202571 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019019945 0 0) 0x7f8a34001490 con 0x7f8a3c06ac80 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a41146640 1 -- 172.21.15.105:0/2669202571 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a28003520 con 0x7f8a3c06ac80 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a41947640 1 -- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c1469e0 msgr2=0x7f8a3c06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a41947640 1 --2- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c1469e0 0x7f8a3c06fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a41947640 1 -- 172.21.15.105:0/2669202571 >> v1:172.21.15.105:6789/0 conn(0x7f8a3c06ac80 legacy=0x7f8a3c0b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a41947640 1 -- 172.21.15.105:0/2669202571 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c06e190 con 0x7f8a3c0ad870 2024-03-08T08:28:57.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.057+0000 7f8a42148640 1 --2- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c1469e0 0x7f8a3c06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:28:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.058+0000 7f8a41947640 1 --2- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 0x7f8a3c06d6f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0097a0 tx=0x7f8a2c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19cc3b06f7653d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.058+0000 7f8a41146640 1 -- 172.21.15.105:0/2669202571 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0315a0 con 0x7f8a3c0ad870 2024-03-08T08:28:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.058+0000 7f8a41146640 1 -- 172.21.15.105:0/2669202571 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c037070 con 0x7f8a3c0ad870 2024-03-08T08:28:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.058+0000 7f8a41146640 1 -- 172.21.15.105:0/2669202571 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c031960 con 0x7f8a3c0ad870 2024-03-08T08:28:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.059+0000 7f8a443ac640 1 -- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 msgr2=0x7f8a3c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.059+0000 7f8a443ac640 1 --2- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 0x7f8a3c06d6f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0097a0 tx=0x7f8a2c02b840 comp rx=0 tx=0).stop 2024-03-08T08:28:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.059+0000 7f8a443ac640 1 -- 172.21.15.105:0/2669202571 shutdown_connections 2024-03-08T08:28:57.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.059+0000 7f8a443ac640 1 --2- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 0x7f8a3c06d6f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.059+0000 7f8a443ac640 1 --2- 172.21.15.105:0/2669202571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c1469e0 0x7f8a3c06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.059+0000 7f8a443ac640 1 -- 172.21.15.105:0/2669202571 >> 172.21.15.105:0/2669202571 conn(0x7f8a3c0b2670 msgr2=0x7f8a3c0b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.059+0000 7f8a443ac640 1 -- 172.21.15.105:0/2669202571 shutdown_connections 2024-03-08T08:28:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.059+0000 7f8a443ac640 1 -- 172.21.15.105:0/2669202571 wait complete. 2024-03-08T08:28:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.060+0000 7f8a443ac640 1 Processor -- start 2024-03-08T08:28:57.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.060+0000 7f8a443ac640 1 -- start start 2024-03-08T08:28:57.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a443ac640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c06ac80 0x7f8a3c0ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a443ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 0x7f8a3c15f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a443ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3c1469e0 0x7f8a3c1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a443ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c06e470 con 0x7f8a3c1469e0 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a443ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0b45d0 con 0x7f8a3c0ad870 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a443ac640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c146dc0 con 0x7f8a3c06ac80 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3c1469e0 0x7f8a3c1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3c1469e0 0x7f8a3c1f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41158/0 (socket says 172.21.15.105:41158) 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 -- 172.21.15.105:0/232385299 learned_addr learned my addr 172.21.15.105:0/232385299 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42148640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c06ac80 0x7f8a3c0ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a41947640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 0x7f8a3c15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 -- 172.21.15.105:0/232385299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c06ac80 msgr2=0x7f8a3c0ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c06ac80 0x7f8a3c0ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 -- 172.21.15.105:0/232385299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 msgr2=0x7f8a3c15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 0x7f8a3c15f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 -- 172.21.15.105:0/232385299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c142e60 con 0x7f8a3c1469e0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a42949640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3c1469e0 0x7f8a3c1f4f20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400cb50 tx=0x7f8a3400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a32ffd640 1 -- 172.21.15.105:0/232385299 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34017070 con 0x7f8a3c1469e0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a32ffd640 1 -- 172.21.15.105:0/232385299 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3400ad40 con 0x7f8a3c1469e0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a443ac640 1 -- 172.21.15.105:0/232385299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c0cdcf0 con 0x7f8a3c1469e0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a32ffd640 1 -- 172.21.15.105:0/232385299 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34012620 con 0x7f8a3c1469e0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.061+0000 7f8a443ac640 1 -- 172.21.15.105:0/232385299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c0ce2b0 con 0x7f8a3c1469e0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.063+0000 7f8a32ffd640 1 -- 172.21.15.105:0/232385299 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a3401a070 con 0x7f8a3c1469e0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.063+0000 7f8a32ffd640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8a2006b4d0 0x7f8a2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.063+0000 7f8a32ffd640 1 -- 172.21.15.105:0/232385299 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a34089fc0 con 0x7f8a3c1469e0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.063+0000 7f8a42148640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8a2006b4d0 0x7f8a2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.064+0000 7f8a42148640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8a2006b4d0 0x7f8a2006d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009560 tx=0x7f8a38009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.064+0000 7f8a46f85740 1 Processor -- start 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.064+0000 7f8a46f85740 1 -- start start 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80f6f8a0 0x555a80f6fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f738f0 0x555a80f75390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a80f9fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a46f85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a80ed9e30 con 0x555a80f738f0 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a46f85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a80e82d20 con 0x555a80f6f8a0 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a46f85740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a80ed5ed0 con 0x555a80f9f630 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a41947640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f738f0 0x555a80f75390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a41947640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f738f0 0x555a80f75390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41162/0 (socket says 172.21.15.105:41162) 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a41947640 1 -- 172.21.15.105:0/1040314957 learned_addr learned my addr 172.21.15.105:0/1040314957 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a42148640 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80f6f8a0 0x555a80f6fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.065+0000 7f8a42949640 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a80f9fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a41947640 1 -- 172.21.15.105:0/1040314957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 msgr2=0x555a80f9fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a41947640 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a80f9fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a41947640 1 -- 172.21.15.105:0/1040314957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80f6f8a0 msgr2=0x555a80f6fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a41947640 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80f6f8a0 0x555a80f6fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a41947640 1 -- 172.21.15.105:0/1040314957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a80bee3f0 con 0x555a80f738f0 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a41947640 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f738f0 0x555a80f75390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0097a0 tx=0x7f8a2c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb99d60a93341e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a1ffff640 1 -- 172.21.15.105:0/1040314957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c031760 con 0x555a80f738f0 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a1ffff640 1 -- 172.21.15.105:0/1040314957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c037040 con 0x555a80f738f0 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.066+0000 7f8a1ffff640 1 -- 172.21.15.105:0/1040314957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c031b00 con 0x555a80f738f0 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 -- 172.21.15.105:0/1040314957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f738f0 msgr2=0x555a80f75390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f738f0 0x555a80f75390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0097a0 tx=0x7f8a2c008030 comp rx=0 tx=0).stop 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 -- 172.21.15.105:0/1040314957 shutdown_connections 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a80f9fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f738f0 0x555a80f75390 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 --2- 172.21.15.105:0/1040314957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80f6f8a0 0x555a80f6fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 -- 172.21.15.105:0/1040314957 >> 172.21.15.105:0/1040314957 conn(0x555a80f71920 msgr2=0x555a80f731e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 -- 172.21.15.105:0/1040314957 shutdown_connections 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 -- 172.21.15.105:0/1040314957 wait complete. 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 -- start start 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.067+0000 7f8a46f85740 1 Processor -- start 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a80fb67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80f9a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80fb9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a46f85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a80f73d80 con 0x555a80f9a150 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a46f85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a80e78390 con 0x555a80fb9b60 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a46f85740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a80f74060 con 0x555a80f9f630 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a46f85740 10 client.0 _ll_register_callbacks cb 0x555a80e95c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a41947640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80f9a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a41947640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80f9a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41174/0 (socket says 172.21.15.105:41174) 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a41947640 1 -- 172.21.15.105:0/3677015823 learned_addr learned my addr 172.21.15.105:0/3677015823 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.068+0000 7f8a42148640 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a80fb67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a42949640 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80fb9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a41947640 1 -- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 msgr2=0x555a80fb67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a41947640 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a80fb67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a41947640 1 -- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 msgr2=0x555a80fb9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a41947640 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80fb9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a41947640 1 -- 172.21.15.105:0/3677015823 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a80cb87d0 con 0x555a80f9a150 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a42949640 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80fb9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a41947640 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80f9a850 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c02be50 tx=0x7f8a2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83ff7527e108620 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/3677015823 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c053060 con 0x555a80f9a150 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/3677015823 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a2c031970 con 0x555a80f9a150 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 1 -- 172.21.15.105:0/3677015823 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a2c037040 con 0x555a80f9a150 2024-03-08T08:28:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/3677015823 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a2c059050 con 0x555a80f9a150 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 10 client.5352 fetch_fsmap learned FSMap version 13 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 1 -- 172.21.15.105:0/3677015823 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x555a80fb5d10 con 0x555a80f9a150 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/3677015823 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c037040 con 0x555a80f9a150 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/3677015823 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8a2c05c020 con 0x555a80f9a150 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 10 client.5352 fetch_fsmap finished waiting for FSMap version 13 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 10 client.5352 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 20 client.5352 populate_metadata read hostname 'smithi105' 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x555a80fa3160 0x555a80fd8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 4 client.5352 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.069+0000 7f8a46f85740 1 -- 172.21.15.105:0/3677015823 --> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a80ee7370 con 0x555a80fa3160 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.070+0000 7f8a42148640 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x555a80fa3160 0x555a80fd8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.070+0000 7f8a1d7fa640 10 client.5352 ms_handle_connect on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.070+0000 7f8a42148640 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x555a80fa3160 0x555a80fd8810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ca9c3468810c16 server_cookie=f6930e4706d44495 in_seq=0 out_seq=0 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.070+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/3677015823 <== mds.0 v2:172.21.15.105:6836/2431299564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a80ee7370 con 0x555a80fa3160 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.070+0000 7f8a1d7fa640 10 client.5352 handle_command_reply: tid=0 2024-03-08T08:28:57.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.070+0000 7f8a46f85740 1 client.5352 shutdown 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.070+0000 7f8a46f85740 1 -- 172.21.15.105:0/3677015823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 msgr2=0x555a80f9a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.070+0000 7f8a46f85740 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80f9a850 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c02be50 tx=0x7f8a2c004730 comp rx=0 tx=0).stop 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 1 -- 172.21.15.105:0/3677015823 shutdown_connections 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x555a80fa3160 0x555a80fd8810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80fb9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80f9a850 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 1 --2- 172.21.15.105:0/3677015823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a80fb67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a1d7fa640 0 client.5352 ms_handle_reset on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 1 -- 172.21.15.105:0/3677015823 >> 172.21.15.105:0/3677015823 conn(0x555a80f6f8a0 msgr2=0x555a80f6e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 1 -- 172.21.15.105:0/3677015823 shutdown_connections 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 1 -- 172.21.15.105:0/3677015823 wait complete. 2024-03-08T08:28:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.071+0000 7f8a46f85740 20 client.5352 trim_cache size 0 max 16384 2024-03-08T08:28:57.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.082+0000 7f8a46f85740 1 Processor -- start 2024-03-08T08:28:57.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.082+0000 7f8a46f85740 1 -- start start 2024-03-08T08:28:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9a150 0x555a80f75470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9f630 0x555a80f9fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80f6e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a46f85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1fff40 con 0x555a80f9f630 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a46f85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1ff0c0 con 0x555a80fb9b60 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a46f85740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1fe590 con 0x555a80f9a150 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a41947640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9f630 0x555a80f9fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a41947640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9f630 0x555a80f9fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41182/0 (socket says 172.21.15.105:41182) 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a41947640 1 -- 172.21.15.105:0/687346035 learned_addr learned my addr 172.21.15.105:0/687346035 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a42148640 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9a150 0x555a80f75470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a42949640 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80f6e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a41947640 1 -- 172.21.15.105:0/687346035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9a150 msgr2=0x555a80f75470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a41947640 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9a150 0x555a80f75470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a41947640 1 -- 172.21.15.105:0/687346035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 msgr2=0x555a80f6e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a41947640 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80f6e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.083+0000 7f8a41947640 1 -- 172.21.15.105:0/687346035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a80fba350 con 0x555a80f9f630 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a41947640 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9f630 0x555a80f9fa10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c031730 tx=0x7f8a2c04b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05762b8f0cba15 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a1f7fe640 1 -- 172.21.15.105:0/687346035 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c031d80 con 0x555a80f9f630 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a1f7fe640 1 -- 172.21.15.105:0/687346035 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c037070 con 0x555a80f9f630 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a1f7fe640 1 -- 172.21.15.105:0/687346035 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c003810 con 0x555a80f9f630 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 -- 172.21.15.105:0/687346035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9f630 msgr2=0x555a80f9fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9f630 0x555a80f9fa10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c031730 tx=0x7f8a2c04b000 comp rx=0 tx=0).stop 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 -- 172.21.15.105:0/687346035 shutdown_connections 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a80f6e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9f630 0x555a80f9fa10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 --2- 172.21.15.105:0/687346035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9a150 0x555a80f75470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 -- 172.21.15.105:0/687346035 >> 172.21.15.105:0/687346035 conn(0x555a80f6f8a0 msgr2=0x555a80fa24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:57.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 -- 172.21.15.105:0/687346035 shutdown_connections 2024-03-08T08:28:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.084+0000 7f8a46f85740 1 -- 172.21.15.105:0/687346035 wait complete. 2024-03-08T08:28:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.085+0000 7f8a46f85740 1 -- start start 2024-03-08T08:28:57.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.085+0000 7f8a46f85740 1 Processor -- start 2024-03-08T08:28:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80fdd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a81025650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a46f85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a81025b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a46f85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1feac0 con 0x555a80f9a150 2024-03-08T08:28:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a46f85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a80fb6cd0 con 0x555a80fb9b60 2024-03-08T08:28:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a46f85740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a81026430 con 0x555a80f9f630 2024-03-08T08:28:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a46f85740 10 client.0 _ll_register_callbacks cb 0x7f8a38009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:28:57.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80fdd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80fdd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41188/0 (socket says 172.21.15.105:41188) 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 -- 172.21.15.105:0/934597099 learned_addr learned my addr 172.21.15.105:0/934597099 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42949640 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a81025b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a41947640 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a81025650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 -- 172.21.15.105:0/934597099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 msgr2=0x555a81025650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a81025650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 -- 172.21.15.105:0/934597099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 msgr2=0x555a81025b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a81025b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 -- 172.21.15.105:0/934597099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a80fb8730 con 0x555a80f9a150 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a42148640 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80fdd0b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x555a80fa35a0 tx=0x7f8a38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6089cfea6d3c82e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/934597099 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3802a280 con 0x555a80f9a150 2024-03-08T08:28:57.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/934597099 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a3802a420 con 0x555a80f9a150 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.086+0000 7f8a46f85740 1 -- 172.21.15.105:0/934597099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a2c037070 con 0x555a80f9a150 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/934597099 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a3803b050 con 0x555a80f9a150 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a46f85740 10 client.5358 fetch_fsmap learned FSMap version 13 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a46f85740 1 -- 172.21.15.105:0/934597099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x555a80fdce60 con 0x555a80f9a150 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/934597099 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c037070 con 0x555a80f9a150 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/934597099 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8a38020040 con 0x555a80f9a150 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a46f85740 10 client.5358 fetch_fsmap finished waiting for FSMap version 13 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a46f85740 10 client.5358 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a46f85740 20 client.5358 populate_metadata read hostname 'smithi105' 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a46f85740 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x555a80f6e5a0 0x555a80f6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a46f85740 4 client.5358 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a46f85740 1 -- 172.21.15.105:0/934597099 --> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x555a80fb96d0 con 0x555a80f6e5a0 2024-03-08T08:28:57.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a41947640 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x555a80f6e5a0 0x555a80f6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a1d7fa640 10 client.5358 ms_handle_connect on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.087+0000 7f8a41947640 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x555a80f6e5a0 0x555a80f6e9e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac7aeb78c2323e9 server_cookie=734222c628eb1db4 in_seq=0 out_seq=0 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a1d7fa640 1 -- 172.21.15.105:0/934597099 <== mds.0 v2:172.21.15.105:6836/2431299564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26925 (crc 0 0 0) 0x555a80fb96d0 con 0x555a80f6e5a0 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a1d7fa640 10 client.5358 handle_command_reply: tid=0 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 client.5358 shutdown 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 -- 172.21.15.105:0/934597099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 msgr2=0x555a80fdd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80fdd0b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x555a80fa35a0 tx=0x7f8a38004730 comp rx=0 tx=0).stop 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 -- 172.21.15.105:0/934597099 shutdown_connections 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x555a80f6e5a0 0x555a80f6e9e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a80fb9b60 0x555a81025b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a80f9f630 0x555a81025650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 --2- 172.21.15.105:0/934597099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555a80f9a150 0x555a80fdd0b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a1d7fa640 0 client.5358 ms_handle_reset on v2:172.21.15.105:6836/2431299564 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 -- 172.21.15.105:0/934597099 >> 172.21.15.105:0/934597099 conn(0x555a80f6f8a0 msgr2=0x555a80f9fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:57.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 -- 172.21.15.105:0/934597099 shutdown_connections 2024-03-08T08:28:57.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 1 -- 172.21.15.105:0/934597099 wait complete. 2024-03-08T08:28:57.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.088+0000 7f8a46f85740 20 client.5358 trim_cache size 0 max 16384 2024-03-08T08:28:57.096 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:28:57.096 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker": [ 2024-03-08T08:28:57.096 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.096 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:57.096 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "0" 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "1" 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:28:57.097 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "2" 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": [ 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.098 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 71, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 65, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 115895, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 149547, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 10, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.014269389, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.002258289, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.015861061, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000610970, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 65, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.090331621, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001389717 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.099 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000383, 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000095 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 112288, 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 12976 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": [ 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 3052, 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 4047, 2024-03-08T08:28:57.100 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 795345, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1912219, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.229750413, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.152504960, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.109927059, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000884198, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4046, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.063510643, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000015697 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6513, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000744340, 2024-03-08T08:28:57.101 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000114 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 96080 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": [ 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1053, 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1042, 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 261978, 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 4535173, 2024-03-08T08:28:57.102 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.064913247, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.027160262, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.014469695, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.019211012, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1041, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.173874729, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000167026 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 25, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000003357, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000134 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.103 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": [ 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": [ 2024-03-08T08:28:57.104 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2048, 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.419536698, 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000204851 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": [ 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.105 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000010937, 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000005468 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": [ 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 1031, 2024-03-08T08:28:57.106 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 1031, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1031, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.565168112, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000548174 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 12, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 1011, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-08T08:28:57.107 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 1011, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 1011, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 1002, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 1003, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 2, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 1061, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 1060, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-08T08:28:57.108 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 858, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 2461, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-08T08:28:57.109 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 1, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 1000, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 1001, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 1000, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 3 2024-03-08T08:28:57.110 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": [ 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-08T08:28:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.112 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_client_metrics": [ 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "5100" 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: "num_clients": 2 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_client_metrics-cephfs": [ 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.113 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "client": "client.4853", 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": "0" 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_hits": 41, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_miss": 0, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_read_latency": 0.000000000, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_write_latency": 0.000000000, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_hits": 0, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_miss": 0, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_files": 0, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_inodes": 1, 2024-03-08T08:28:57.114 INFO:teuthology.orchestra.run.smithi105.stdout: "pinned_icaps": 1, 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "total_inodes": 1, 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_ops": 0, 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_size": 0, 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_ops": 0, 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_size": 0 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "client": "client.5089", 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": "0" 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_hits": 8742, 2024-03-08T08:28:57.115 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_miss": 19, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_read_latency": 0.000000000, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_write_latency": 0.000000000, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_hits": 872, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_miss": 0, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_files": 2, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_inodes": 873, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "pinned_icaps": 873, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "total_inodes": 873, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_ops": 0, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_size": 0, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_ops": 870, 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_size": 2500 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.116 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": [ 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 2008, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 2008, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 3, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 3, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-08T08:28:57.117 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 8387435, 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1005, 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.373472396, 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001366639 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": [ 2024-03-08T08:28:57.118 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 1014, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1014, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 13, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 13, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 1011, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 1011, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 1003, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 1003, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-03-08T08:28:57.119 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 55552, 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215392 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": [ 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 1032, 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 1031, 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 1003, 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-08T08:28:57.120 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1000, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.558526483, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000558526 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 29, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.003770316, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000130010 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.121 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.122 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.123 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002613698, 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002613698 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.124 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.125 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000257615, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000257615 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-08T08:28:57.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": [ 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.130 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 1007, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 503, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 12, 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": [ 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.131 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T08:28:57.132 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T08:28:57.133 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 258618, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 547, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 1848, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 21, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-08T08:28:57.134 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5312, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 75, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 1555173, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 17698, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-08T08:28:57.135 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": [ 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 1048, 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 4271410, 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 31, 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 1048, 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-08T08:28:57.136 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1048, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.062959787, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001968473 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1048, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1076 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 1048, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 1048, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-08T08:28:57.137 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 11, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 1005, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 8, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-08T08:28:57.138 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 22, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-08T08:28:57.139 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 33, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 1, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 7, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 7, 2024-03-08T08:28:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 23, 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 1 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": [ 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-08T08:28:57.141 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": [ 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": [ 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.142 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.143 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 4170, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 793644, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 4170, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4170, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 793644, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.144 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": [ 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:57.145 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 1048, 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 4271410, 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1048, 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 4271410, 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": [ 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.146 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 1048, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1048, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1048, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1048, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.147 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": [ 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:28:57.148 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x557e3b40e6e0": [ 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:28:57.149 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 2008, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 4193131, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 2008, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1005, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 4193131, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:28:57.150 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:28:57.151 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-08T08:28:57.151 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:28:57.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 -- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8a2006b4d0 msgr2=0x7f8a2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8a2006b4d0 0x7f8a2006d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009560 tx=0x7f8a38009190 comp rx=0 tx=0).stop 2024-03-08T08:28:57.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 -- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3c1469e0 msgr2=0x7f8a3c1f4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:28:57.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3c1469e0 0x7f8a3c1f4f20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400cb50 tx=0x7f8a3400a430 comp rx=0 tx=0).stop 2024-03-08T08:28:57.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 -- 172.21.15.105:0/232385299 shutdown_connections 2024-03-08T08:28:57.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8a2006b4d0 0x7f8a2006d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a3c1469e0 0x7f8a3c1f4f20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3c0ad870 0x7f8a3c15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 --2- 172.21.15.105:0/232385299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3c06ac80 0x7f8a3c0ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:28:57.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 -- 172.21.15.105:0/232385299 >> 172.21.15.105:0/232385299 conn(0x7f8a3c0b2670 msgr2=0x7f8a3c144850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:28:57.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 -- 172.21.15.105:0/232385299 shutdown_connections 2024-03-08T08:28:57.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:28:57.089+0000 7f8a443ac640 1 -- 172.21.15.105:0/232385299 wait complete. 2024-03-08T08:28:57.153 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:57.153 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:57.169 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_0", '"'"'r'"'"') as f: 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:57.170 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:57.171 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:57.171 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:57.212 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_0 2024-03-08T08:28:57.229 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_0 2024-03-08T08:28:57.229 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:57.230 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627777 Links: 1 2024-03-08T08:28:57.230 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:57.230 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:57.230 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.413441864 +0000 2024-03-08T08:28:57.230 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.416441806 +0000 2024-03-08T08:28:57.230 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.416441806 +0000 2024-03-08T08:28:57.230 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.413441864 +0000 2024-03-08T08:28:57.230 INFO:tasks.cephfs.mount:File test_dir/test_file_0 became visible from 1 after 0s 2024-03-08T08:28:57.230 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:57.230 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:57.277 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:57.285 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:57.285 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:57.285 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:57.285 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:57.285 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:57.285 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.285 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:57.285 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_1", '"'"'r'"'"') as f: 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:57.286 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:57.327 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_1 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_1 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627778 Links: 1 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.425441630 +0000 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.425441630 +0000 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.425441630 +0000 2024-03-08T08:28:57.344 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.425441630 +0000 2024-03-08T08:28:57.344 INFO:tasks.cephfs.mount:File test_dir/test_file_1 became visible from 1 after 0s 2024-03-08T08:28:57.344 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:57.345 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:57.391 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_2", '"'"'r'"'"') as f: 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:57.400 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:57.442 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_2 2024-03-08T08:28:57.458 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_2 2024-03-08T08:28:57.458 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:57.458 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627779 Links: 1 2024-03-08T08:28:57.458 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:57.458 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:57.459 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.429441552 +0000 2024-03-08T08:28:57.459 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.429441552 +0000 2024-03-08T08:28:57.459 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.429441552 +0000 2024-03-08T08:28:57.459 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.429441552 +0000 2024-03-08T08:28:57.459 INFO:tasks.cephfs.mount:File test_dir/test_file_2 became visible from 1 after 0s 2024-03-08T08:28:57.459 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:57.459 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:57.507 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:57.513 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:57.513 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:57.513 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:57.513 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:57.513 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:57.513 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_3", '"'"'r'"'"') as f: 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:57.514 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:57.555 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_3 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_3 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627780 Links: 1 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.433441474 +0000 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.433441474 +0000 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.433441474 +0000 2024-03-08T08:28:57.572 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.433441474 +0000 2024-03-08T08:28:57.572 INFO:tasks.cephfs.mount:File test_dir/test_file_3 became visible from 1 after 0s 2024-03-08T08:28:57.572 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:57.572 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:57.620 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_4", '"'"'r'"'"') as f: 2024-03-08T08:28:57.627 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:57.628 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:57.628 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:57.628 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:57.628 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:57.628 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:57.628 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:57.628 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:57.628 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:57.669 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_4 2024-03-08T08:28:57.685 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_4 2024-03-08T08:28:57.685 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:57.686 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627781 Links: 1 2024-03-08T08:28:57.686 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:57.686 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:57.686 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.438441377 +0000 2024-03-08T08:28:57.686 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.438441377 +0000 2024-03-08T08:28:57.686 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.438441377 +0000 2024-03-08T08:28:57.686 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.438441377 +0000 2024-03-08T08:28:57.686 INFO:tasks.cephfs.mount:File test_dir/test_file_4 became visible from 1 after 0s 2024-03-08T08:28:57.686 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:57.686 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:57.734 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_5", '"'"'r'"'"') as f: 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:57.741 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:57.742 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:57.742 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:57.742 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:57.742 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:57.742 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:57.742 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:57.783 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_5 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_5 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627782 Links: 1 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.445441241 +0000 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.445441241 +0000 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.445441241 +0000 2024-03-08T08:28:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.445441241 +0000 2024-03-08T08:28:57.800 INFO:tasks.cephfs.mount:File test_dir/test_file_5 became visible from 1 after 0s 2024-03-08T08:28:57.800 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:57.801 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:57.847 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:57.855 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:57.855 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:57.855 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:57.855 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:57.855 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:57.855 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.855 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:57.855 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_6", '"'"'r'"'"') as f: 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:57.856 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:57.898 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_6 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_6 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627783 Links: 1 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.449441163 +0000 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.449441163 +0000 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.449441163 +0000 2024-03-08T08:28:57.915 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.449441163 +0000 2024-03-08T08:28:57.915 INFO:tasks.cephfs.mount:File test_dir/test_file_6 became visible from 1 after 0s 2024-03-08T08:28:57.916 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:57.916 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:57.964 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:57.970 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:57.970 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:57.970 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:57.970 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:57.970 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:57.970 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.970 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:57.970 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_7", '"'"'r'"'"') as f: 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:57.971 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.012 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_7 2024-03-08T08:28:58.028 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_7 2024-03-08T08:28:58.028 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.028 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627784 Links: 1 2024-03-08T08:28:58.029 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.029 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.029 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.453441085 +0000 2024-03-08T08:28:58.029 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.453441085 +0000 2024-03-08T08:28:58.029 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.453441085 +0000 2024-03-08T08:28:58.029 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.453441085 +0000 2024-03-08T08:28:58.029 INFO:tasks.cephfs.mount:File test_dir/test_file_7 became visible from 1 after 0s 2024-03-08T08:28:58.029 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.029 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.077 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:58.084 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:58.084 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:58.084 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:58.084 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:58.084 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:58.084 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.084 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_8", '"'"'r'"'"') as f: 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:58.085 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.127 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_8 2024-03-08T08:28:58.143 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_8 2024-03-08T08:28:58.143 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.143 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627785 Links: 1 2024-03-08T08:28:58.143 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.143 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.143 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.456441026 +0000 2024-03-08T08:28:58.143 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.456441026 +0000 2024-03-08T08:28:58.144 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.456441026 +0000 2024-03-08T08:28:58.144 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.456441026 +0000 2024-03-08T08:28:58.144 INFO:tasks.cephfs.mount:File test_dir/test_file_8 became visible from 1 after 0s 2024-03-08T08:28:58.144 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.144 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.191 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:58.198 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:58.198 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:58.198 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:58.198 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:58.198 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:58.198 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_9", '"'"'r'"'"') as f: 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:58.199 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.241 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_9 2024-03-08T08:28:58.257 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_9 2024-03-08T08:28:58.257 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.257 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627786 Links: 1 2024-03-08T08:28:58.257 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.257 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.257 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.459440968 +0000 2024-03-08T08:28:58.258 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.459440968 +0000 2024-03-08T08:28:58.258 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.459440968 +0000 2024-03-08T08:28:58.258 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.459440968 +0000 2024-03-08T08:28:58.258 INFO:tasks.cephfs.mount:File test_dir/test_file_9 became visible from 1 after 0s 2024-03-08T08:28:58.258 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.258 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.305 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:58.312 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:58.312 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:58.312 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:58.312 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:58.312 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:58.312 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.312 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_10", '"'"'r'"'"') as f: 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:58.313 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.354 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_10 2024-03-08T08:28:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_10 2024-03-08T08:28:58.370 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.370 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627787 Links: 1 2024-03-08T08:28:58.371 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.371 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.371 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.465440851 +0000 2024-03-08T08:28:58.371 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.465440851 +0000 2024-03-08T08:28:58.371 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.465440851 +0000 2024-03-08T08:28:58.371 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.465440851 +0000 2024-03-08T08:28:58.371 INFO:tasks.cephfs.mount:File test_dir/test_file_10 became visible from 1 after 0s 2024-03-08T08:28:58.371 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.371 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.418 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:58.426 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:58.426 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:58.426 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:58.426 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:58.426 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_11", '"'"'r'"'"') as f: 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:58.427 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.468 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_11 2024-03-08T08:28:58.486 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_11 2024-03-08T08:28:58.486 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.486 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627788 Links: 1 2024-03-08T08:28:58.486 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.486 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.486 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.469440773 +0000 2024-03-08T08:28:58.486 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.469440773 +0000 2024-03-08T08:28:58.486 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.469440773 +0000 2024-03-08T08:28:58.487 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.469440773 +0000 2024-03-08T08:28:58.487 INFO:tasks.cephfs.mount:File test_dir/test_file_11 became visible from 1 after 0s 2024-03-08T08:28:58.487 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.487 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.531 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.541 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_12", '"'"'r'"'"') as f: 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:58.542 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.583 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_12 2024-03-08T08:28:58.601 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_12 2024-03-08T08:28:58.601 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.602 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627789 Links: 1 2024-03-08T08:28:58.602 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.602 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.602 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.474440676 +0000 2024-03-08T08:28:58.602 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.475440656 +0000 2024-03-08T08:28:58.602 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.475440656 +0000 2024-03-08T08:28:58.602 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.474440676 +0000 2024-03-08T08:28:58.602 INFO:tasks.cephfs.mount:File test_dir/test_file_12 became visible from 1 after 0s 2024-03-08T08:28:58.602 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.602 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.646 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_13", '"'"'r'"'"') as f: 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:58.657 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:58.658 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:58.658 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:58.658 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:58.658 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:58.658 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:58.658 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.699 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_13 2024-03-08T08:28:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_13 2024-03-08T08:28:58.717 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.717 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627790 Links: 1 2024-03-08T08:28:58.717 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.717 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.718 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.479440578 +0000 2024-03-08T08:28:58.718 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.479440578 +0000 2024-03-08T08:28:58.718 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.479440578 +0000 2024-03-08T08:28:58.718 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.479440578 +0000 2024-03-08T08:28:58.718 INFO:tasks.cephfs.mount:File test_dir/test_file_13 became visible from 1 after 0s 2024-03-08T08:28:58.718 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.718 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.762 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:58.772 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:58.772 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:58.772 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:58.772 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:58.772 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:58.772 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.772 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:58.772 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_14", '"'"'r'"'"') as f: 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:58.773 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.815 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_14 2024-03-08T08:28:58.835 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_14 2024-03-08T08:28:58.835 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.835 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627791 Links: 1 2024-03-08T08:28:58.835 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.835 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.836 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.484440481 +0000 2024-03-08T08:28:58.836 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.484440481 +0000 2024-03-08T08:28:58.836 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.484440481 +0000 2024-03-08T08:28:58.836 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.484440481 +0000 2024-03-08T08:28:58.836 INFO:tasks.cephfs.mount:File test_dir/test_file_14 became visible from 1 after 0s 2024-03-08T08:28:58.836 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.836 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.878 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:58.890 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:58.890 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:58.890 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:58.890 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_15", '"'"'r'"'"') as f: 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:58.891 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:58.932 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_15 2024-03-08T08:28:58.952 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_15 2024-03-08T08:28:58.953 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:58.953 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627792 Links: 1 2024-03-08T08:28:58.953 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:58.953 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:58.953 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.489440383 +0000 2024-03-08T08:28:58.953 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.489440383 +0000 2024-03-08T08:28:58.953 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.489440383 +0000 2024-03-08T08:28:58.953 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.489440383 +0000 2024-03-08T08:28:58.953 INFO:tasks.cephfs.mount:File test_dir/test_file_15 became visible from 1 after 0s 2024-03-08T08:28:58.953 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:58.953 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:58.995 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.011 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_16", '"'"'r'"'"') as f: 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.012 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.053 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_16 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_16 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627793 Links: 1 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.493440305 +0000 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.493440305 +0000 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.493440305 +0000 2024-03-08T08:28:59.074 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.493440305 +0000 2024-03-08T08:28:59.074 INFO:tasks.cephfs.mount:File test_dir/test_file_16 became visible from 1 after 0s 2024-03-08T08:28:59.075 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.075 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:59.116 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.129 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.129 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.129 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.129 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.129 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.129 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.129 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_17", '"'"'r'"'"') as f: 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.130 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.171 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_17 2024-03-08T08:28:59.191 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_17 2024-03-08T08:28:59.191 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.191 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627794 Links: 1 2024-03-08T08:28:59.191 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.191 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.191 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.496440247 +0000 2024-03-08T08:28:59.191 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.496440247 +0000 2024-03-08T08:28:59.191 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.496440247 +0000 2024-03-08T08:28:59.192 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.496440247 +0000 2024-03-08T08:28:59.192 INFO:tasks.cephfs.mount:File test_dir/test_file_17 became visible from 1 after 0s 2024-03-08T08:28:59.192 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.192 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:59.233 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.246 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.246 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.246 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_18", '"'"'r'"'"') as f: 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.247 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.279 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.281 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_18 2024-03-08T08:28:59.301 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_18 2024-03-08T08:28:59.301 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.301 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627795 Links: 1 2024-03-08T08:28:59.302 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.302 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.302 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.499440188 +0000 2024-03-08T08:28:59.302 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.499440188 +0000 2024-03-08T08:28:59.302 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.499440188 +0000 2024-03-08T08:28:59.302 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.499440188 +0000 2024-03-08T08:28:59.302 INFO:tasks.cephfs.mount:File test_dir/test_file_18 became visible from 1 after 0s 2024-03-08T08:28:59.302 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.302 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:59.344 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.357 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.357 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.357 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.357 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.357 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.357 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.357 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_19", '"'"'r'"'"') as f: 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.358 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.393 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.395 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_19 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_19 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627796 Links: 1 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.502440130 +0000 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.502440130 +0000 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.502440130 +0000 2024-03-08T08:28:59.415 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.502440130 +0000 2024-03-08T08:28:59.415 INFO:tasks.cephfs.mount:File test_dir/test_file_19 became visible from 1 after 0s 2024-03-08T08:28:59.416 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.416 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:59.458 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.470 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_20", '"'"'r'"'"') as f: 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.509 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.511 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_20 2024-03-08T08:28:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_20 2024-03-08T08:28:59.531 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.531 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627797 Links: 1 2024-03-08T08:28:59.531 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.531 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.531 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.508440013 +0000 2024-03-08T08:28:59.532 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.508440013 +0000 2024-03-08T08:28:59.532 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.508440013 +0000 2024-03-08T08:28:59.532 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.508440013 +0000 2024-03-08T08:28:59.532 INFO:tasks.cephfs.mount:File test_dir/test_file_20 became visible from 1 after 0s 2024-03-08T08:28:59.532 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.532 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:59.574 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.586 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_21", '"'"'r'"'"') as f: 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.587 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.588 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.622 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.624 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_21 2024-03-08T08:28:59.644 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_21 2024-03-08T08:28:59.644 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.645 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627798 Links: 1 2024-03-08T08:28:59.645 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.645 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.645 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.513439916 +0000 2024-03-08T08:28:59.645 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.513439916 +0000 2024-03-08T08:28:59.645 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.513439916 +0000 2024-03-08T08:28:59.645 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.513439916 +0000 2024-03-08T08:28:59.645 INFO:tasks.cephfs.mount:File test_dir/test_file_21 became visible from 1 after 0s 2024-03-08T08:28:59.645 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.645 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:59.686 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.700 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.700 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.700 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.700 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.700 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_22", '"'"'r'"'"') as f: 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.701 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.736 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.737 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_22 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_22 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627799 Links: 1 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.516439857 +0000 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.516439857 +0000 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.516439857 +0000 2024-03-08T08:28:59.758 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.516439857 +0000 2024-03-08T08:28:59.758 INFO:tasks.cephfs.mount:File test_dir/test_file_22 became visible from 1 after 0s 2024-03-08T08:28:59.759 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.759 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:59.801 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.813 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.813 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.813 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.813 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.813 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.813 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_23", '"'"'r'"'"') as f: 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.814 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.849 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.851 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_23 2024-03-08T08:28:59.869 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_23 2024-03-08T08:28:59.869 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.869 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627800 Links: 1 2024-03-08T08:28:59.869 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.869 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.869 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.525439682 +0000 2024-03-08T08:28:59.870 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.525439682 +0000 2024-03-08T08:28:59.870 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.525439682 +0000 2024-03-08T08:28:59.870 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.525439682 +0000 2024-03-08T08:28:59.870 INFO:tasks.cephfs.mount:File test_dir/test_file_23 became visible from 1 after 0s 2024-03-08T08:28:59.870 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.870 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:28:59.914 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.924 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:28:59.924 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_24", '"'"'r'"'"') as f: 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:28:59.925 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:28:59.966 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:28:59.968 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_24 2024-03-08T08:28:59.987 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_24 2024-03-08T08:28:59.987 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:28:59.988 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627801 Links: 1 2024-03-08T08:28:59.988 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:28:59.988 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:28:59.988 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.528439624 +0000 2024-03-08T08:28:59.988 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.528439624 +0000 2024-03-08T08:28:59.988 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.528439624 +0000 2024-03-08T08:28:59.988 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.528439624 +0000 2024-03-08T08:28:59.988 INFO:tasks.cephfs.mount:File test_dir/test_file_24 became visible from 1 after 0s 2024-03-08T08:28:59.988 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:28:59.988 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.030 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.044 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.044 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_25", '"'"'r'"'"') as f: 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.045 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.079 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.081 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_25 2024-03-08T08:29:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_25 2024-03-08T08:29:00.099 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:00.100 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627802 Links: 1 2024-03-08T08:29:00.100 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:00.100 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:00.100 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.532439545 +0000 2024-03-08T08:29:00.100 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.532439545 +0000 2024-03-08T08:29:00.100 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.532439545 +0000 2024-03-08T08:29:00.100 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.532439545 +0000 2024-03-08T08:29:00.100 INFO:tasks.cephfs.mount:File test_dir/test_file_25 became visible from 1 after 0s 2024-03-08T08:29:00.100 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:00.100 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.144 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.156 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.156 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.156 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.156 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.156 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.156 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_26", '"'"'r'"'"') as f: 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.157 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.194 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.196 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_26 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_26 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627803 Links: 1 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.538439429 +0000 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.538439429 +0000 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.538439429 +0000 2024-03-08T08:29:00.216 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.538439429 +0000 2024-03-08T08:29:00.217 INFO:tasks.cephfs.mount:File test_dir/test_file_26 became visible from 1 after 0s 2024-03-08T08:29:00.217 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:00.217 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.259 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.271 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_27", '"'"'r'"'"') as f: 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.272 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.273 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.273 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.273 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.307 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.309 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_27 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_27 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627804 Links: 1 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.544439312 +0000 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.544439312 +0000 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.544439312 +0000 2024-03-08T08:29:00.328 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.544439312 +0000 2024-03-08T08:29:00.329 INFO:tasks.cephfs.mount:File test_dir/test_file_27 became visible from 1 after 0s 2024-03-08T08:29:00.329 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:00.329 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.372 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.383 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.383 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.383 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.383 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.383 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.383 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.383 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_28", '"'"'r'"'"') as f: 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.384 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.420 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.422 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_28 2024-03-08T08:29:00.440 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_28 2024-03-08T08:29:00.440 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:00.440 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627805 Links: 1 2024-03-08T08:29:00.440 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:00.440 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:00.440 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.547439253 +0000 2024-03-08T08:29:00.440 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.547439253 +0000 2024-03-08T08:29:00.441 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.547439253 +0000 2024-03-08T08:29:00.441 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.547439253 +0000 2024-03-08T08:29:00.441 INFO:tasks.cephfs.mount:File test_dir/test_file_28 became visible from 1 after 0s 2024-03-08T08:29:00.441 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:00.441 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.485 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.495 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_29", '"'"'r'"'"') as f: 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.496 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.533 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.535 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_29 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_29 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627806 Links: 1 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.553439136 +0000 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.553439136 +0000 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.553439136 +0000 2024-03-08T08:29:00.554 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.553439136 +0000 2024-03-08T08:29:00.554 INFO:tasks.cephfs.mount:File test_dir/test_file_29 became visible from 1 after 0s 2024-03-08T08:29:00.554 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:00.555 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.598 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_30", '"'"'r'"'"') as f: 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.609 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.610 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.610 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.610 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.610 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.610 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.610 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.610 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.649 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.650 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_30 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_30 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627807 Links: 1 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.558439039 +0000 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.558439039 +0000 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.558439039 +0000 2024-03-08T08:29:00.669 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.558439039 +0000 2024-03-08T08:29:00.669 INFO:tasks.cephfs.mount:File test_dir/test_file_30 became visible from 1 after 0s 2024-03-08T08:29:00.670 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:00.670 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.713 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.724 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.724 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.724 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.724 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.724 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.724 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_31", '"'"'r'"'"') as f: 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.725 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.764 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.765 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_31 2024-03-08T08:29:00.783 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_31 2024-03-08T08:29:00.784 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:00.784 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627808 Links: 1 2024-03-08T08:29:00.784 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:00.784 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:00.784 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.562438961 +0000 2024-03-08T08:29:00.784 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.562438961 +0000 2024-03-08T08:29:00.784 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.562438961 +0000 2024-03-08T08:29:00.784 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.562438961 +0000 2024-03-08T08:29:00.784 INFO:tasks.cephfs.mount:File test_dir/test_file_31 became visible from 1 after 0s 2024-03-08T08:29:00.784 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:00.784 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.828 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_32", '"'"'r'"'"') as f: 2024-03-08T08:29:00.839 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.840 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.840 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.840 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.840 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.840 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.840 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.840 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.840 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.879 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.881 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_32 2024-03-08T08:29:00.899 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_32 2024-03-08T08:29:00.899 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:00.899 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627809 Links: 1 2024-03-08T08:29:00.899 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:00.900 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:00.900 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.565438903 +0000 2024-03-08T08:29:00.900 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.565438903 +0000 2024-03-08T08:29:00.900 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.565438903 +0000 2024-03-08T08:29:00.900 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.565438903 +0000 2024-03-08T08:29:00.900 INFO:tasks.cephfs.mount:File test_dir/test_file_32 became visible from 1 after 0s 2024-03-08T08:29:00.900 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:00.900 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:00.944 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:00.955 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_33", '"'"'r'"'"') as f: 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:00.956 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:00.997 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_33 2024-03-08T08:29:00.998 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_33 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627810 Links: 1 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.568438844 +0000 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.568438844 +0000 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.568438844 +0000 2024-03-08T08:29:01.016 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.568438844 +0000 2024-03-08T08:29:01.017 INFO:tasks.cephfs.mount:File test_dir/test_file_33 became visible from 1 after 0s 2024-03-08T08:29:01.017 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.017 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.060 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.071 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_34", '"'"'r'"'"') as f: 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.072 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:01.113 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_34 2024-03-08T08:29:01.118 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_34 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627811 Links: 1 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.571438786 +0000 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.571438786 +0000 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.571438786 +0000 2024-03-08T08:29:01.132 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.571438786 +0000 2024-03-08T08:29:01.132 INFO:tasks.cephfs.mount:File test_dir/test_file_34 became visible from 1 after 0s 2024-03-08T08:29:01.132 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.133 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.176 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_35", '"'"'r'"'"') as f: 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.187 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.188 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.188 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.188 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:01.229 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_35 2024-03-08T08:29:01.235 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_35 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627812 Links: 1 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.576438688 +0000 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.576438688 +0000 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.576438688 +0000 2024-03-08T08:29:01.248 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.576438688 +0000 2024-03-08T08:29:01.249 INFO:tasks.cephfs.mount:File test_dir/test_file_35 became visible from 1 after 0s 2024-03-08T08:29:01.249 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.249 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.281 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.292 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.295 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.295 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.295 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.295 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_36", '"'"'r'"'"') as f: 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.296 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:01.337 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_36 2024-03-08T08:29:01.346 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_36 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627813 Links: 1 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.580438610 +0000 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.580438610 +0000 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.580438610 +0000 2024-03-08T08:29:01.356 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.580438610 +0000 2024-03-08T08:29:01.356 INFO:tasks.cephfs.mount:File test_dir/test_file_36 became visible from 1 after 0s 2024-03-08T08:29:01.356 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.357 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.395 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.400 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_37", '"'"'r'"'"') as f: 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.409 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.410 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.410 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.410 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.410 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.410 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.410 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:01.451 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_37 2024-03-08T08:29:01.459 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_37 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627814 Links: 1 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.583438552 +0000 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.583438552 +0000 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.583438552 +0000 2024-03-08T08:29:01.470 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.583438552 +0000 2024-03-08T08:29:01.470 INFO:tasks.cephfs.mount:File test_dir/test_file_37 became visible from 1 after 0s 2024-03-08T08:29:01.471 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.471 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.511 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.514 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.525 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.525 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.525 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_38", '"'"'r'"'"') as f: 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.526 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:01.568 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_38 2024-03-08T08:29:01.576 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.587 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_38 2024-03-08T08:29:01.587 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.587 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627815 Links: 1 2024-03-08T08:29:01.587 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.588 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.588 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.586438493 +0000 2024-03-08T08:29:01.588 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.586438493 +0000 2024-03-08T08:29:01.588 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.586438493 +0000 2024-03-08T08:29:01.588 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.586438493 +0000 2024-03-08T08:29:01.588 INFO:tasks.cephfs.mount:File test_dir/test_file_38 became visible from 1 after 0s 2024-03-08T08:29:01.588 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.588 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.624 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.631 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.638 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.638 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.638 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.638 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.638 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.638 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.638 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_39", '"'"'r'"'"') as f: 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.639 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:01.680 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_39 2024-03-08T08:29:01.688 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_39 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627816 Links: 1 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.589438435 +0000 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.589438435 +0000 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.589438435 +0000 2024-03-08T08:29:01.700 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.589438435 +0000 2024-03-08T08:29:01.700 INFO:tasks.cephfs.mount:File test_dir/test_file_39 became visible from 1 after 0s 2024-03-08T08:29:01.700 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.700 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.738 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.743 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_40", '"'"'r'"'"') as f: 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.753 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.754 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.754 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.754 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:01.795 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_40 2024-03-08T08:29:01.802 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.814 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_40 2024-03-08T08:29:01.815 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.815 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627817 Links: 1 2024-03-08T08:29:01.815 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.815 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.815 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.593438357 +0000 2024-03-08T08:29:01.815 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.593438357 +0000 2024-03-08T08:29:01.815 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.593438357 +0000 2024-03-08T08:29:01.815 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.593438357 +0000 2024-03-08T08:29:01.815 INFO:tasks.cephfs.mount:File test_dir/test_file_40 became visible from 1 after 0s 2024-03-08T08:29:01.815 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.815 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.851 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.858 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_41", '"'"'r'"'"') as f: 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.866 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.867 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.867 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.867 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.867 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.867 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.867 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:01.908 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_41 2024-03-08T08:29:01.916 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.927 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_41 2024-03-08T08:29:01.928 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:01.928 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627818 Links: 1 2024-03-08T08:29:01.928 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:01.928 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:01.928 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.596438299 +0000 2024-03-08T08:29:01.928 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.596438299 +0000 2024-03-08T08:29:01.928 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.596438299 +0000 2024-03-08T08:29:01.928 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.596438299 +0000 2024-03-08T08:29:01.928 INFO:tasks.cephfs.mount:File test_dir/test_file_41 became visible from 1 after 0s 2024-03-08T08:29:01.928 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:01.928 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:01.968 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.972 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:01.982 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:01.982 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:01.982 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:01.982 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:01.982 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:01.982 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.982 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_42", '"'"'r'"'"') as f: 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:01.983 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.024 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_42 2024-03-08T08:29:02.032 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.044 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_42 2024-03-08T08:29:02.044 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:02.044 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627819 Links: 1 2024-03-08T08:29:02.044 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:02.044 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:02.044 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.599438240 +0000 2024-03-08T08:29:02.045 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.600438221 +0000 2024-03-08T08:29:02.045 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.600438221 +0000 2024-03-08T08:29:02.045 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.599438240 +0000 2024-03-08T08:29:02.045 INFO:tasks.cephfs.mount:File test_dir/test_file_42 became visible from 1 after 0s 2024-03-08T08:29:02.045 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:02.045 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:02.081 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.088 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.095 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:02.095 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:02.095 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:02.095 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:02.095 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:02.095 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.095 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_43", '"'"'r'"'"') as f: 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:02.096 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.138 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_43 2024-03-08T08:29:02.145 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.156 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_43 2024-03-08T08:29:02.156 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:02.156 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627820 Links: 1 2024-03-08T08:29:02.156 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:02.156 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:02.156 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.603438162 +0000 2024-03-08T08:29:02.156 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.603438162 +0000 2024-03-08T08:29:02.156 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.603438162 +0000 2024-03-08T08:29:02.157 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.603438162 +0000 2024-03-08T08:29:02.157 INFO:tasks.cephfs.mount:File test_dir/test_file_43 became visible from 1 after 0s 2024-03-08T08:29:02.157 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:02.157 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:02.196 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.200 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.210 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:02.210 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:02.210 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:02.210 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:02.210 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:02.210 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.210 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:02.210 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_44", '"'"'r'"'"') as f: 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:02.211 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.252 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_44 2024-03-08T08:29:02.261 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.271 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_44 2024-03-08T08:29:02.271 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:02.271 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627821 Links: 1 2024-03-08T08:29:02.271 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:02.271 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:02.271 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.606438104 +0000 2024-03-08T08:29:02.271 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.606438104 +0000 2024-03-08T08:29:02.271 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.606438104 +0000 2024-03-08T08:29:02.272 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.606438104 +0000 2024-03-08T08:29:02.272 INFO:tasks.cephfs.mount:File test_dir/test_file_44 became visible from 1 after 0s 2024-03-08T08:29:02.272 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:02.272 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:02.309 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.315 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.323 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:02.323 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:02.323 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:02.323 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:02.323 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_45", '"'"'r'"'"') as f: 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.365 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_45 2024-03-08T08:29:02.374 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.386 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_45 2024-03-08T08:29:02.386 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:02.386 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627822 Links: 1 2024-03-08T08:29:02.386 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:02.387 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:02.387 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.610438026 +0000 2024-03-08T08:29:02.387 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.610438026 +0000 2024-03-08T08:29:02.387 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.610438026 +0000 2024-03-08T08:29:02.387 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.610438026 +0000 2024-03-08T08:29:02.387 INFO:tasks.cephfs.mount:File test_dir/test_file_45 became visible from 1 after 0s 2024-03-08T08:29:02.387 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:02.387 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:02.421 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.428 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_46", '"'"'r'"'"') as f: 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:02.436 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:02.437 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:02.437 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:02.437 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:02.437 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:02.437 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:02.437 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.479 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_46 2024-03-08T08:29:02.487 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_46 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627823 Links: 1 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.614437948 +0000 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.614437948 +0000 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.614437948 +0000 2024-03-08T08:29:02.500 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.614437948 +0000 2024-03-08T08:29:02.500 INFO:tasks.cephfs.mount:File test_dir/test_file_46 became visible from 1 after 0s 2024-03-08T08:29:02.500 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:02.500 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:02.535 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.541 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.549 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:02.549 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:02.549 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_47", '"'"'r'"'"') as f: 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:02.550 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.592 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_47 2024-03-08T08:29:02.600 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.613 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_47 2024-03-08T08:29:02.613 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:02.613 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627824 Links: 1 2024-03-08T08:29:02.613 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:02.613 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:02.613 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.618437870 +0000 2024-03-08T08:29:02.614 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.618437870 +0000 2024-03-08T08:29:02.614 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.618437870 +0000 2024-03-08T08:29:02.614 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.618437870 +0000 2024-03-08T08:29:02.614 INFO:tasks.cephfs.mount:File test_dir/test_file_47 became visible from 1 after 0s 2024-03-08T08:29:02.614 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:02.614 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:02.649 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.656 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.663 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:02.663 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:02.663 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:02.663 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_48", '"'"'r'"'"') as f: 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:02.664 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:02.665 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:02.665 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:02.665 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.705 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_48 2024-03-08T08:29:02.715 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_48 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627825 Links: 1 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.621437811 +0000 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.621437811 +0000 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.621437811 +0000 2024-03-08T08:29:02.726 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.621437811 +0000 2024-03-08T08:29:02.726 INFO:tasks.cephfs.mount:File test_dir/test_file_48 became visible from 1 after 0s 2024-03-08T08:29:02.727 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:02.727 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:02.766 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.768 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_49", '"'"'r'"'"') as f: 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:02.781 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:02.782 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:02.782 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:02.782 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:02.782 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:02.782 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:02.782 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:02.782 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.823 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_49 2024-03-08T08:29:02.830 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.845 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_49 2024-03-08T08:29:02.845 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:02.845 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627826 Links: 1 2024-03-08T08:29:02.845 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:02.845 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:02.846 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.624437753 +0000 2024-03-08T08:29:02.846 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.624437753 +0000 2024-03-08T08:29:02.846 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.624437753 +0000 2024-03-08T08:29:02.846 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.624437753 +0000 2024-03-08T08:29:02.846 INFO:tasks.cephfs.mount:File test_dir/test_file_49 became visible from 1 after 0s 2024-03-08T08:29:02.846 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:02.846 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:02.881 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.890 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:02.895 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:02.895 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:02.895 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:02.895 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:02.895 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:02.895 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.895 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:02.895 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_50", '"'"'r'"'"') as f: 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:02.896 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:02.938 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_50 2024-03-08T08:29:02.946 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.000 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.063 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.120 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.178 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.237 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.266 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_50 2024-03-08T08:29:03.267 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:03.268 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627827 Links: 1 2024-03-08T08:29:03.268 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:03.268 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:03.268 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.627437694 +0000 2024-03-08T08:29:03.268 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.627437694 +0000 2024-03-08T08:29:03.268 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.627437694 +0000 2024-03-08T08:29:03.268 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.627437694 +0000 2024-03-08T08:29:03.268 INFO:tasks.cephfs.mount:File test_dir/test_file_50 became visible from 1 after 0s 2024-03-08T08:29:03.268 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:03.268 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:03.283 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.294 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.299 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:03.299 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:03.299 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:03.299 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:03.299 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:03.299 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.299 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_51", '"'"'r'"'"') as f: 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:03.300 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:03.341 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_51 2024-03-08T08:29:03.348 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_51 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627828 Links: 1 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.631437617 +0000 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.631437617 +0000 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.631437617 +0000 2024-03-08T08:29:03.356 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.631437617 +0000 2024-03-08T08:29:03.357 INFO:tasks.cephfs.mount:File test_dir/test_file_51 became visible from 1 after 0s 2024-03-08T08:29:03.357 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:03.357 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:03.397 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.402 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.404 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.411 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:03.411 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:03.411 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:03.411 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:03.411 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_52", '"'"'r'"'"') as f: 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:03.412 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:03.453 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_52 2024-03-08T08:29:03.461 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_52 2024-03-08T08:29:03.468 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:03.468 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627829 Links: 1 2024-03-08T08:29:03.468 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:03.468 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:03.468 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.637437500 +0000 2024-03-08T08:29:03.468 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.637437500 +0000 2024-03-08T08:29:03.469 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.637437500 +0000 2024-03-08T08:29:03.469 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.637437500 +0000 2024-03-08T08:29:03.469 INFO:tasks.cephfs.mount:File test_dir/test_file_52 became visible from 1 after 0s 2024-03-08T08:29:03.469 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:03.469 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:03.513 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.516 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.516 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_53", '"'"'r'"'"') as f: 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:03.524 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:03.525 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:03.525 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:03.525 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:03.525 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:03.525 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:03.525 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:03.525 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:03.566 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_53 2024-03-08T08:29:03.578 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.581 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_53 2024-03-08T08:29:03.581 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:03.581 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627830 Links: 1 2024-03-08T08:29:03.582 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:03.582 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:03.582 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.640437441 +0000 2024-03-08T08:29:03.582 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.640437441 +0000 2024-03-08T08:29:03.582 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.640437441 +0000 2024-03-08T08:29:03.582 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.640437441 +0000 2024-03-08T08:29:03.582 INFO:tasks.cephfs.mount:File test_dir/test_file_53 became visible from 1 after 0s 2024-03-08T08:29:03.582 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:03.582 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:03.626 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.629 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.633 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.637 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:03.637 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:03.637 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:03.637 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:03.637 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:03.637 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_54", '"'"'r'"'"') as f: 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:03.638 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:03.679 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_54 2024-03-08T08:29:03.690 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_54 2024-03-08T08:29:03.694 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:03.694 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627831 Links: 1 2024-03-08T08:29:03.694 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:03.694 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:03.694 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.643437383 +0000 2024-03-08T08:29:03.694 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.643437383 +0000 2024-03-08T08:29:03.694 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.643437383 +0000 2024-03-08T08:29:03.695 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.643437383 +0000 2024-03-08T08:29:03.695 INFO:tasks.cephfs.mount:File test_dir/test_file_54 became visible from 1 after 0s 2024-03-08T08:29:03.695 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:03.695 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:03.740 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.742 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.745 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.749 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:03.749 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:03.749 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:03.749 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:03.749 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_55", '"'"'r'"'"') as f: 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:03.750 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:03.792 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_55 2024-03-08T08:29:03.804 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_55 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627832 Links: 1 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.647437305 +0000 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.647437305 +0000 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.647437305 +0000 2024-03-08T08:29:03.807 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.647437305 +0000 2024-03-08T08:29:03.807 INFO:tasks.cephfs.mount:File test_dir/test_file_55 became visible from 1 after 0s 2024-03-08T08:29:03.807 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:03.808 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:03.853 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.854 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.860 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.862 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:03.862 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:03.862 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:03.862 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:03.862 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:03.862 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.862 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_56", '"'"'r'"'"') as f: 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:03.863 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:03.904 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_56 2024-03-08T08:29:03.918 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.919 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_56 2024-03-08T08:29:03.920 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:03.920 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627833 Links: 1 2024-03-08T08:29:03.920 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:03.920 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:03.920 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.650437246 +0000 2024-03-08T08:29:03.920 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.650437246 +0000 2024-03-08T08:29:03.920 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.650437246 +0000 2024-03-08T08:29:03.920 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.650437246 +0000 2024-03-08T08:29:03.920 INFO:tasks.cephfs.mount:File test_dir/test_file_56 became visible from 1 after 0s 2024-03-08T08:29:03.920 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:03.920 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:03.968 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.970 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.973 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:03.975 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_57", '"'"'r'"'"') as f: 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:03.976 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.017 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_57 2024-03-08T08:29:04.032 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_57 2024-03-08T08:29:04.032 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.032 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627834 Links: 1 2024-03-08T08:29:04.033 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.033 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.033 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.653437188 +0000 2024-03-08T08:29:04.033 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.653437188 +0000 2024-03-08T08:29:04.033 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.653437188 +0000 2024-03-08T08:29:04.033 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.653437188 +0000 2024-03-08T08:29:04.033 INFO:tasks.cephfs.mount:File test_dir/test_file_57 became visible from 1 after 0s 2024-03-08T08:29:04.033 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.033 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.034 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_58", '"'"'r'"'"') as f: 2024-03-08T08:29:04.048 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.049 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.049 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.049 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.049 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.049 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.049 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.049 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.049 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.081 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.083 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_58 2024-03-08T08:29:04.083 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.090 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.098 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_58 2024-03-08T08:29:04.098 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.098 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627835 Links: 1 2024-03-08T08:29:04.098 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.098 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.098 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.656437130 +0000 2024-03-08T08:29:04.098 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.656437130 +0000 2024-03-08T08:29:04.098 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.656437130 +0000 2024-03-08T08:29:04.099 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.656437130 +0000 2024-03-08T08:29:04.099 INFO:tasks.cephfs.mount:File test_dir/test_file_58 became visible from 1 after 0s 2024-03-08T08:29:04.099 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.099 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.145 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.147 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.153 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.153 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.153 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.153 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.153 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.153 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_59", '"'"'r'"'"') as f: 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.154 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.195 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_59 2024-03-08T08:29:04.198 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.201 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_59 2024-03-08T08:29:04.210 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.210 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627836 Links: 1 2024-03-08T08:29:04.211 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.211 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.211 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.660437052 +0000 2024-03-08T08:29:04.211 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.660437052 +0000 2024-03-08T08:29:04.211 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.660437052 +0000 2024-03-08T08:29:04.211 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.660437052 +0000 2024-03-08T08:29:04.211 INFO:tasks.cephfs.mount:File test_dir/test_file_59 became visible from 1 after 0s 2024-03-08T08:29:04.211 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.211 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.258 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.263 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.266 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.266 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.266 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.266 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.266 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.266 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_60", '"'"'r'"'"') as f: 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.267 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.308 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_60 2024-03-08T08:29:04.311 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.318 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_60 2024-03-08T08:29:04.323 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.323 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627837 Links: 1 2024-03-08T08:29:04.323 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.323 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.324 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.663436993 +0000 2024-03-08T08:29:04.324 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.663436993 +0000 2024-03-08T08:29:04.324 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.663436993 +0000 2024-03-08T08:29:04.324 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.663436993 +0000 2024-03-08T08:29:04.324 INFO:tasks.cephfs.mount:File test_dir/test_file_60 became visible from 1 after 0s 2024-03-08T08:29:04.324 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.324 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.371 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.376 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.378 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.378 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.378 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.378 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.378 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.378 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_61", '"'"'r'"'"') as f: 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.379 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.420 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_61 2024-03-08T08:29:04.423 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.430 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.435 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_61 2024-03-08T08:29:04.435 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.436 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627838 Links: 1 2024-03-08T08:29:04.436 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.436 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.436 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.666436935 +0000 2024-03-08T08:29:04.436 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.666436935 +0000 2024-03-08T08:29:04.436 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.666436935 +0000 2024-03-08T08:29:04.436 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.666436935 +0000 2024-03-08T08:29:04.436 INFO:tasks.cephfs.mount:File test_dir/test_file_61 became visible from 1 after 0s 2024-03-08T08:29:04.436 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.436 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.483 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.489 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.491 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.491 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.491 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.491 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.491 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.491 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_62", '"'"'r'"'"') as f: 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.492 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.533 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_62 2024-03-08T08:29:04.537 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.549 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_62 2024-03-08T08:29:04.549 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.549 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627839 Links: 1 2024-03-08T08:29:04.550 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.550 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.550 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.669436876 +0000 2024-03-08T08:29:04.550 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.669436876 +0000 2024-03-08T08:29:04.550 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.669436876 +0000 2024-03-08T08:29:04.550 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.669436876 +0000 2024-03-08T08:29:04.550 INFO:tasks.cephfs.mount:File test_dir/test_file_62 became visible from 1 after 0s 2024-03-08T08:29:04.550 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.550 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.597 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.602 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.605 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.605 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.605 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.605 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.605 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_63", '"'"'r'"'"') as f: 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.647 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_63 2024-03-08T08:29:04.651 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.657 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_63 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627840 Links: 1 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.672436818 +0000 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.672436818 +0000 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.672436818 +0000 2024-03-08T08:29:04.663 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.672436818 +0000 2024-03-08T08:29:04.664 INFO:tasks.cephfs.mount:File test_dir/test_file_63 became visible from 1 after 0s 2024-03-08T08:29:04.664 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.664 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.710 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.717 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.718 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.718 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.718 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.718 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.718 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.718 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_64", '"'"'r'"'"') as f: 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.719 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.760 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_64 2024-03-08T08:29:04.768 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.770 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_64 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627841 Links: 1 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.675436759 +0000 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.675436759 +0000 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.675436759 +0000 2024-03-08T08:29:04.775 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.675436759 +0000 2024-03-08T08:29:04.775 INFO:tasks.cephfs.mount:File test_dir/test_file_64 became visible from 1 after 0s 2024-03-08T08:29:04.776 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.776 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.823 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.830 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.830 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.830 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.830 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.830 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_65", '"'"'r'"'"') as f: 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.831 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.832 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.834 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_65 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_65 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627842 Links: 1 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.678436701 +0000 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.678436701 +0000 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.678436701 +0000 2024-03-08T08:29:04.849 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.678436701 +0000 2024-03-08T08:29:04.850 INFO:tasks.cephfs.mount:File test_dir/test_file_65 became visible from 1 after 0s 2024-03-08T08:29:04.850 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.850 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:04.883 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.889 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.899 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:04.899 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:04.899 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:04.899 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:04.899 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:04.899 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.899 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_66", '"'"'r'"'"') as f: 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:04.900 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:04.900 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.941 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_66 2024-03-08T08:29:04.948 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_66 2024-03-08T08:29:04.956 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:04.956 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627843 Links: 1 2024-03-08T08:29:04.956 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:04.957 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:04.957 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.682436623 +0000 2024-03-08T08:29:04.957 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.682436623 +0000 2024-03-08T08:29:04.957 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.682436623 +0000 2024-03-08T08:29:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.682436623 +0000 2024-03-08T08:29:04.957 INFO:tasks.cephfs.mount:File test_dir/test_file_66 became visible from 1 after 0s 2024-03-08T08:29:04.957 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:04.957 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.000 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_67", '"'"'r'"'"') as f: 2024-03-08T08:29:05.012 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.013 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.013 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.013 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.013 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.013 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.013 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.013 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.013 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.054 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_67 2024-03-08T08:29:05.064 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.069 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_67 2024-03-08T08:29:05.070 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.070 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627844 Links: 1 2024-03-08T08:29:05.070 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.070 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.070 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.685436565 +0000 2024-03-08T08:29:05.070 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.685436565 +0000 2024-03-08T08:29:05.070 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.685436565 +0000 2024-03-08T08:29:05.070 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.685436565 +0000 2024-03-08T08:29:05.070 INFO:tasks.cephfs.mount:File test_dir/test_file_67 became visible from 1 after 0s 2024-03-08T08:29:05.070 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.070 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.117 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.122 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_68", '"'"'r'"'"') as f: 2024-03-08T08:29:05.125 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.126 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.126 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.126 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.126 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.126 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.126 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.126 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.126 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.167 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_68 2024-03-08T08:29:05.180 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.182 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_68 2024-03-08T08:29:05.182 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.182 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627845 Links: 1 2024-03-08T08:29:05.182 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.182 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.182 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.688436506 +0000 2024-03-08T08:29:05.183 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.688436506 +0000 2024-03-08T08:29:05.183 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.688436506 +0000 2024-03-08T08:29:05.183 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.688436506 +0000 2024-03-08T08:29:05.183 INFO:tasks.cephfs.mount:File test_dir/test_file_68 became visible from 1 after 0s 2024-03-08T08:29:05.183 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.183 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.230 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_69", '"'"'r'"'"') as f: 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.237 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.238 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.238 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.238 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.238 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.238 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.238 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.239 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.240 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_69 2024-03-08T08:29:05.255 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_69 2024-03-08T08:29:05.255 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.256 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627846 Links: 1 2024-03-08T08:29:05.256 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.256 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.256 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.692436428 +0000 2024-03-08T08:29:05.256 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.692436428 +0000 2024-03-08T08:29:05.256 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.692436428 +0000 2024-03-08T08:29:05.256 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.692436428 +0000 2024-03-08T08:29:05.256 INFO:tasks.cephfs.mount:File test_dir/test_file_69 became visible from 1 after 0s 2024-03-08T08:29:05.256 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.256 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.268 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.282 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.282 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.282 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.282 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.282 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.282 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_70", '"'"'r'"'"') as f: 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.283 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.285 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.286 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_70 2024-03-08T08:29:05.296 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.301 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_70 2024-03-08T08:29:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.302 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627847 Links: 1 2024-03-08T08:29:05.302 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.302 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.302 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.695436370 +0000 2024-03-08T08:29:05.302 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.695436370 +0000 2024-03-08T08:29:05.302 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.695436370 +0000 2024-03-08T08:29:05.302 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.695436370 +0000 2024-03-08T08:29:05.302 INFO:tasks.cephfs.mount:File test_dir/test_file_70 became visible from 1 after 0s 2024-03-08T08:29:05.302 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.302 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.304 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.318 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.318 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.318 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.318 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.318 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_71", '"'"'r'"'"') as f: 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.319 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.349 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.350 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.351 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_71 2024-03-08T08:29:05.366 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_71 2024-03-08T08:29:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.367 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627848 Links: 1 2024-03-08T08:29:05.367 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.367 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.367 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.699436292 +0000 2024-03-08T08:29:05.367 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.699436292 +0000 2024-03-08T08:29:05.367 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.699436292 +0000 2024-03-08T08:29:05.367 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.699436292 +0000 2024-03-08T08:29:05.367 INFO:tasks.cephfs.mount:File test_dir/test_file_71 became visible from 1 after 0s 2024-03-08T08:29:05.367 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.367 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.399 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.404 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.406 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.413 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.413 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.413 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.413 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.413 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.413 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_72", '"'"'r'"'"') as f: 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.415 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.417 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_72 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_72 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627849 Links: 1 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.707436136 +0000 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.707436136 +0000 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.707436136 +0000 2024-03-08T08:29:05.432 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.707436136 +0000 2024-03-08T08:29:05.432 INFO:tasks.cephfs.mount:File test_dir/test_file_72 became visible from 1 after 0s 2024-03-08T08:29:05.433 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.433 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.463 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_73", '"'"'r'"'"') as f: 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.477 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.478 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.478 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.478 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.478 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.478 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.478 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.478 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.480 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.481 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_73 2024-03-08T08:29:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_73 2024-03-08T08:29:05.496 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.496 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627850 Links: 1 2024-03-08T08:29:05.496 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.496 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.497 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.710436077 +0000 2024-03-08T08:29:05.497 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.710436077 +0000 2024-03-08T08:29:05.497 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.710436077 +0000 2024-03-08T08:29:05.497 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.710436077 +0000 2024-03-08T08:29:05.497 INFO:tasks.cephfs.mount:File test_dir/test_file_73 became visible from 1 after 0s 2024-03-08T08:29:05.497 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.497 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.515 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.518 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.518 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.529 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.529 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.529 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_74", '"'"'r'"'"') as f: 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.544 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.546 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_74 2024-03-08T08:29:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_74 2024-03-08T08:29:05.561 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.561 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627851 Links: 1 2024-03-08T08:29:05.561 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.562 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.562 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.713436019 +0000 2024-03-08T08:29:05.562 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.713436019 +0000 2024-03-08T08:29:05.562 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.713436019 +0000 2024-03-08T08:29:05.562 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.713436019 +0000 2024-03-08T08:29:05.562 INFO:tasks.cephfs.mount:File test_dir/test_file_74 became visible from 1 after 0s 2024-03-08T08:29:05.562 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.562 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.580 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_75", '"'"'r'"'"') as f: 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.595 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.596 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.596 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.609 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.611 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_75 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_75 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627852 Links: 1 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.716435960 +0000 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.716435960 +0000 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.716435960 +0000 2024-03-08T08:29:05.626 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.716435960 +0000 2024-03-08T08:29:05.626 INFO:tasks.cephfs.mount:File test_dir/test_file_75 became visible from 1 after 0s 2024-03-08T08:29:05.626 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.627 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.635 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.642 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.642 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.642 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.642 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.642 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.642 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.642 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_76", '"'"'r'"'"') as f: 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.643 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.675 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.677 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_76 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_76 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627853 Links: 1 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.719435902 +0000 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.719435902 +0000 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.719435902 +0000 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.719435902 +0000 2024-03-08T08:29:05.692 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.693 INFO:tasks.cephfs.mount:File test_dir/test_file_76 became visible from 1 after 0s 2024-03-08T08:29:05.693 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.693 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.742 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.744 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.747 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.748 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_77", '"'"'r'"'"') as f: 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.749 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.790 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_77 2024-03-08T08:29:05.805 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_77 2024-03-08T08:29:05.806 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.806 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627854 Links: 1 2024-03-08T08:29:05.806 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.806 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.806 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.722435844 +0000 2024-03-08T08:29:05.806 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.722435844 +0000 2024-03-08T08:29:05.806 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.722435844 +0000 2024-03-08T08:29:05.806 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.722435844 +0000 2024-03-08T08:29:05.806 INFO:tasks.cephfs.mount:File test_dir/test_file_77 became visible from 1 after 0s 2024-03-08T08:29:05.807 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.807 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.807 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.854 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.855 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.856 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.861 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.861 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.861 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.861 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.861 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.861 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.861 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_78", '"'"'r'"'"') as f: 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.862 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.863 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.864 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_78 2024-03-08T08:29:05.880 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_78 2024-03-08T08:29:05.880 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.880 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627855 Links: 1 2024-03-08T08:29:05.880 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.881 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.881 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.725435785 +0000 2024-03-08T08:29:05.881 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.725435785 +0000 2024-03-08T08:29:05.881 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.725435785 +0000 2024-03-08T08:29:05.881 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.725435785 +0000 2024-03-08T08:29:05.881 INFO:tasks.cephfs.mount:File test_dir/test_file_78 became visible from 1 after 0s 2024-03-08T08:29:05.881 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.881 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:05.920 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.927 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_79", '"'"'r'"'"') as f: 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:05.935 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:05.936 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:05.936 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:05.970 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.972 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_79 2024-03-08T08:29:05.972 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.974 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:05.987 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_79 2024-03-08T08:29:05.987 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:05.987 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627856 Links: 1 2024-03-08T08:29:05.987 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:05.987 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:05.987 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.728435727 +0000 2024-03-08T08:29:05.987 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.728435727 +0000 2024-03-08T08:29:05.988 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.728435727 +0000 2024-03-08T08:29:05.988 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.728435727 +0000 2024-03-08T08:29:05.988 INFO:tasks.cephfs.mount:File test_dir/test_file_79 became visible from 1 after 0s 2024-03-08T08:29:05.988 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:05.988 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.035 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.035 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_80", '"'"'r'"'"') as f: 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.042 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.043 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.043 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.043 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.043 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.043 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.043 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.083 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.085 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_80 2024-03-08T08:29:06.085 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.091 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_80 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627857 Links: 1 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.732435649 +0000 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.732435649 +0000 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.732435649 +0000 2024-03-08T08:29:06.100 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.732435649 +0000 2024-03-08T08:29:06.101 INFO:tasks.cephfs.mount:File test_dir/test_file_80 became visible from 1 after 0s 2024-03-08T08:29:06.101 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.101 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.147 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.147 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.149 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.155 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.155 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.155 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.155 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.155 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.155 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.155 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.155 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_81", '"'"'r'"'"') as f: 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.156 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.197 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_81 2024-03-08T08:29:06.199 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.204 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.212 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_81 2024-03-08T08:29:06.212 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627858 Links: 1 2024-03-08T08:29:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.735435590 +0000 2024-03-08T08:29:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.735435590 +0000 2024-03-08T08:29:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.735435590 +0000 2024-03-08T08:29:06.213 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.735435590 +0000 2024-03-08T08:29:06.213 INFO:tasks.cephfs.mount:File test_dir/test_file_81 became visible from 1 after 0s 2024-03-08T08:29:06.213 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.213 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.260 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.260 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.265 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.268 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_82", '"'"'r'"'"') as f: 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.269 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.270 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.270 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.311 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_82 2024-03-08T08:29:06.313 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.319 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.326 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_82 2024-03-08T08:29:06.326 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.326 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627859 Links: 1 2024-03-08T08:29:06.326 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.326 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.327 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.738435532 +0000 2024-03-08T08:29:06.327 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.738435532 +0000 2024-03-08T08:29:06.327 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.738435532 +0000 2024-03-08T08:29:06.327 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.738435532 +0000 2024-03-08T08:29:06.327 INFO:tasks.cephfs.mount:File test_dir/test_file_82 became visible from 1 after 0s 2024-03-08T08:29:06.327 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.327 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.372 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.374 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.377 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.381 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_83", '"'"'r'"'"') as f: 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.382 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.423 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_83 2024-03-08T08:29:06.425 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.432 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.438 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_83 2024-03-08T08:29:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.439 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627860 Links: 1 2024-03-08T08:29:06.439 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.439 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.439 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.743435434 +0000 2024-03-08T08:29:06.439 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.743435434 +0000 2024-03-08T08:29:06.439 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.743435434 +0000 2024-03-08T08:29:06.439 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.743435434 +0000 2024-03-08T08:29:06.439 INFO:tasks.cephfs.mount:File test_dir/test_file_83 became visible from 1 after 0s 2024-03-08T08:29:06.439 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.439 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.485 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.486 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.491 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_84", '"'"'r'"'"') as f: 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.494 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.495 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.495 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.495 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.495 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.495 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.495 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.495 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.536 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_84 2024-03-08T08:29:06.539 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.545 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_84 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627861 Links: 1 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.746435376 +0000 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.746435376 +0000 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.746435376 +0000 2024-03-08T08:29:06.552 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.746435376 +0000 2024-03-08T08:29:06.552 INFO:tasks.cephfs.mount:File test_dir/test_file_84 became visible from 1 after 0s 2024-03-08T08:29:06.552 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.553 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.599 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.599 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.604 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.607 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_85", '"'"'r'"'"') as f: 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.608 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.609 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.609 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.650 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_85 2024-03-08T08:29:06.653 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.659 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.665 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_85 2024-03-08T08:29:06.666 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.666 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627862 Links: 1 2024-03-08T08:29:06.666 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.666 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.666 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.749435318 +0000 2024-03-08T08:29:06.666 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.749435318 +0000 2024-03-08T08:29:06.666 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.749435318 +0000 2024-03-08T08:29:06.666 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.749435318 +0000 2024-03-08T08:29:06.666 INFO:tasks.cephfs.mount:File test_dir/test_file_85 became visible from 1 after 0s 2024-03-08T08:29:06.666 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.666 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.712 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.713 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.719 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_86", '"'"'r'"'"') as f: 2024-03-08T08:29:06.721 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.722 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.722 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.722 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.722 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.722 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.722 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.722 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.722 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.763 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_86 2024-03-08T08:29:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.772 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.778 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_86 2024-03-08T08:29:06.778 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.778 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627863 Links: 1 2024-03-08T08:29:06.778 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.778 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.778 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.752435259 +0000 2024-03-08T08:29:06.778 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.752435259 +0000 2024-03-08T08:29:06.779 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.752435259 +0000 2024-03-08T08:29:06.779 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.752435259 +0000 2024-03-08T08:29:06.779 INFO:tasks.cephfs.mount:File test_dir/test_file_86 became visible from 1 after 0s 2024-03-08T08:29:06.779 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.779 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.825 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.826 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.833 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.833 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.833 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.833 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.833 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.833 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.833 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.833 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_87", '"'"'r'"'"') as f: 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.834 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.835 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.836 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_87 2024-03-08T08:29:06.851 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_87 2024-03-08T08:29:06.851 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.851 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627864 Links: 1 2024-03-08T08:29:06.852 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.852 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.852 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.755435201 +0000 2024-03-08T08:29:06.852 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.755435201 +0000 2024-03-08T08:29:06.852 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.755435201 +0000 2024-03-08T08:29:06.852 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.755435201 +0000 2024-03-08T08:29:06.852 INFO:tasks.cephfs.mount:File test_dir/test_file_87 became visible from 1 after 0s 2024-03-08T08:29:06.852 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.852 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:06.885 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.891 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.899 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.900 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_88", '"'"'r'"'"') as f: 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:06.901 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:06.902 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.943 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_88 2024-03-08T08:29:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_88 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627865 Links: 1 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.759435123 +0000 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.759435123 +0000 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.759435123 +0000 2024-03-08T08:29:06.958 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.759435123 +0000 2024-03-08T08:29:06.959 INFO:tasks.cephfs.mount:File test_dir/test_file_88 became visible from 1 after 0s 2024-03-08T08:29:06.959 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:06.959 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.005 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.007 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.013 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_89", '"'"'r'"'"') as f: 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.014 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.055 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_89 2024-03-08T08:29:07.066 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.070 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_89 2024-03-08T08:29:07.071 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.071 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627866 Links: 1 2024-03-08T08:29:07.071 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.071 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.071 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.762435064 +0000 2024-03-08T08:29:07.071 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.762435064 +0000 2024-03-08T08:29:07.071 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.762435064 +0000 2024-03-08T08:29:07.071 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.762435064 +0000 2024-03-08T08:29:07.071 INFO:tasks.cephfs.mount:File test_dir/test_file_89 became visible from 1 after 0s 2024-03-08T08:29:07.071 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.071 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.117 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.119 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.124 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_90", '"'"'r'"'"') as f: 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.127 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.128 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.128 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.128 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.128 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.128 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.128 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.170 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_90 2024-03-08T08:29:07.204 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.205 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_90 2024-03-08T08:29:07.205 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.205 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627867 Links: 1 2024-03-08T08:29:07.205 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.205 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.205 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.765435006 +0000 2024-03-08T08:29:07.206 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.765435006 +0000 2024-03-08T08:29:07.206 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.765435006 +0000 2024-03-08T08:29:07.206 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.765435006 +0000 2024-03-08T08:29:07.206 INFO:tasks.cephfs.mount:File test_dir/test_file_90 became visible from 1 after 0s 2024-03-08T08:29:07.206 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.206 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.232 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.233 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.241 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.247 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_91", '"'"'r'"'"') as f: 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.248 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.270 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.272 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_91 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_91 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627868 Links: 1 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.768434948 +0000 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.769434928 +0000 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.769434928 +0000 2024-03-08T08:29:07.287 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.768434948 +0000 2024-03-08T08:29:07.287 INFO:tasks.cephfs.mount:File test_dir/test_file_91 became visible from 1 after 0s 2024-03-08T08:29:07.288 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.288 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.288 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.298 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.305 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.312 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.312 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_92", '"'"'r'"'"') as f: 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.313 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.314 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.335 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.337 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_92 2024-03-08T08:29:07.351 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.352 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.352 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_92 2024-03-08T08:29:07.352 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.353 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627869 Links: 1 2024-03-08T08:29:07.353 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.353 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.353 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.774434831 +0000 2024-03-08T08:29:07.353 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.774434831 +0000 2024-03-08T08:29:07.353 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.774434831 +0000 2024-03-08T08:29:07.353 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.774434831 +0000 2024-03-08T08:29:07.353 INFO:tasks.cephfs.mount:File test_dir/test_file_92 became visible from 1 after 0s 2024-03-08T08:29:07.353 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.353 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.400 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.400 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.406 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.408 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.408 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.408 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.408 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_93", '"'"'r'"'"') as f: 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.409 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.409 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.417 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.418 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_93 2024-03-08T08:29:07.433 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_93 2024-03-08T08:29:07.434 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.434 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627870 Links: 1 2024-03-08T08:29:07.434 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.434 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.434 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.778434752 +0000 2024-03-08T08:29:07.434 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.778434752 +0000 2024-03-08T08:29:07.434 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.778434752 +0000 2024-03-08T08:29:07.434 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.778434752 +0000 2024-03-08T08:29:07.434 INFO:tasks.cephfs.mount:File test_dir/test_file_93 became visible from 1 after 0s 2024-03-08T08:29:07.434 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.435 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.465 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.479 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.479 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.479 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.479 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.479 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.479 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_94", '"'"'r'"'"') as f: 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.480 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.482 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_94 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_94 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627871 Links: 1 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.781434694 +0000 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.781434694 +0000 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.781434694 +0000 2024-03-08T08:29:07.498 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.781434694 +0000 2024-03-08T08:29:07.499 INFO:tasks.cephfs.mount:File test_dir/test_file_94 became visible from 1 after 0s 2024-03-08T08:29:07.499 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.499 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.517 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.520 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.520 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.531 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.531 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.531 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.531 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_95", '"'"'r'"'"') as f: 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.532 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.546 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.546 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.547 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_95 2024-03-08T08:29:07.562 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_95 2024-03-08T08:29:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.563 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627872 Links: 1 2024-03-08T08:29:07.563 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.563 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.563 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.785434616 +0000 2024-03-08T08:29:07.563 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.785434616 +0000 2024-03-08T08:29:07.563 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.785434616 +0000 2024-03-08T08:29:07.563 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.785434616 +0000 2024-03-08T08:29:07.563 INFO:tasks.cephfs.mount:File test_dir/test_file_95 became visible from 1 after 0s 2024-03-08T08:29:07.563 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.563 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.582 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.596 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_96", '"'"'r'"'"') as f: 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.597 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.611 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.611 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.612 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_96 2024-03-08T08:29:07.627 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_96 2024-03-08T08:29:07.628 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.628 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627873 Links: 1 2024-03-08T08:29:07.628 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.628 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.628 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.788434558 +0000 2024-03-08T08:29:07.628 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.788434558 +0000 2024-03-08T08:29:07.628 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.788434558 +0000 2024-03-08T08:29:07.628 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.788434558 +0000 2024-03-08T08:29:07.628 INFO:tasks.cephfs.mount:File test_dir/test_file_96 became visible from 1 after 0s 2024-03-08T08:29:07.628 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.628 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.630 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.633 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.637 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_97", '"'"'r'"'"') as f: 2024-03-08T08:29:07.644 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.645 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.645 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.645 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.645 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.645 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.645 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.645 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.645 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.676 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.678 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_97 2024-03-08T08:29:07.678 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.693 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_97 2024-03-08T08:29:07.693 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.693 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627874 Links: 1 2024-03-08T08:29:07.693 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.693 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.693 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.791434499 +0000 2024-03-08T08:29:07.693 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.791434499 +0000 2024-03-08T08:29:07.694 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.791434499 +0000 2024-03-08T08:29:07.694 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.791434499 +0000 2024-03-08T08:29:07.694 INFO:tasks.cephfs.mount:File test_dir/test_file_97 became visible from 1 after 0s 2024-03-08T08:29:07.694 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.694 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_98", '"'"'r'"'"') as f: 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.709 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.710 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.710 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.710 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.710 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.710 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.741 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.743 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_98 2024-03-08T08:29:07.743 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.744 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.746 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.749 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.759 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_98 2024-03-08T08:29:07.759 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.759 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627875 Links: 1 2024-03-08T08:29:07.759 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.759 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.759 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.794434441 +0000 2024-03-08T08:29:07.759 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.794434441 +0000 2024-03-08T08:29:07.760 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.794434441 +0000 2024-03-08T08:29:07.760 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.794434441 +0000 2024-03-08T08:29:07.760 INFO:tasks.cephfs.mount:File test_dir/test_file_98 became visible from 1 after 0s 2024-03-08T08:29:07.760 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:07.760 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:07.806 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.808 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> stdin-killer -- python3 -c ' 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> import fcntl 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> import sys 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> import time 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_99", '"'"'r'"'"') as f: 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> while True: 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> print("open_background: keeping file open", file=sys.stderr) 2024-03-08T08:29:07.815 DEBUG:teuthology.orchestra.run.smithi144:> try: 2024-03-08T08:29:07.816 DEBUG:teuthology.orchestra.run.smithi144:> if os.read(0, 4096) == b"": 2024-03-08T08:29:07.816 DEBUG:teuthology.orchestra.run.smithi144:> break 2024-03-08T08:29:07.816 DEBUG:teuthology.orchestra.run.smithi144:> except BlockingIOError: 2024-03-08T08:29:07.816 DEBUG:teuthology.orchestra.run.smithi144:> pass 2024-03-08T08:29:07.816 DEBUG:teuthology.orchestra.run.smithi144:> time.sleep(2) 2024-03-08T08:29:07.816 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-03-08T08:29:07.856 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.858 DEBUG:teuthology.orchestra.run.smithi144:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_99 2024-03-08T08:29:07.859 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.859 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.864 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_99 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 2ah/42d Inode: 1099511627876 Links: 1 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-08 08:28:53.797434382 +0000 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-08 08:28:53.797434382 +0000 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-08 08:28:53.797434382 +0000 2024-03-08T08:29:07.875 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-08 08:28:53.797434382 +0000 2024-03-08T08:29:07.875 INFO:tasks.cephfs.mount:File test_dir/test_file_99 became visible from 1 after 0s 2024-03-08T08:29:07.876 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-08T08:29:07.876 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-03-08T08:29:07.922 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.922 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.929 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.972 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.974 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.977 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:07.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.990+0000 7f8b85a64640 1 Processor -- start 2024-03-08T08:29:07.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.990+0000 7f8b85a64640 1 -- start start 2024-03-08T08:29:07.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.990+0000 7f8b85a64640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8b80161850 0x7f8b8006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:07.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.990+0000 7f8b85a64640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800d48a0 con 0x7f8b80161470 2024-03-08T08:29:07.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.990+0000 7f8b85a64640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800d4b80 con 0x7f8b8006bcb0 2024-03-08T08:29:07.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.990+0000 7f8b85a64640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8006c170 con 0x7f8b80161850 2024-03-08T08:29:07.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.990+0000 7f8b7f7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8b80161470 0x7f8b8006b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56990/0 (socket says 172.21.15.105:56990) 2024-03-08T08:29:07.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.990+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/231273138 learned_addr learned my addr 172.21.15.105:0/231273138 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/231273138 >> v2:172.21.15.144:3300/0 conn(0x7f8b80161850 0x7f8b8006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162131952 0 0) 0x7f8b800d48a0 con 0x7f8b80161470 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b6c003520 con 0x7f8b80161470 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475888125 0 0) 0x7f8b800d4b80 con 0x7f8b8006bcb0 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b800d48a0 con 0x7f8b8006bcb0 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2413128587 0 0) 0x7f8b6c003520 con 0x7f8b80161470 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 >> v2:172.21.15.144:3300/0 conn(0x7f8b80161850 msgr2=0x7f8b8006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 --2- 172.21.15.105:0/231273138 >> v2:172.21.15.144:3300/0 conn(0x7f8b80161850 0x7f8b8006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 >> v1:172.21.15.144:6789/0 conn(0x7f8b8006bcb0 legacy=0x7f8b800b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8006c450 con 0x7f8b80161470 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f8b70001670 con 0x7f8b80161470 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/231273138 >> v2:172.21.15.144:3300/0 conn(0x7f8b80161850 0x7f8b8006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 >> v1:172.21.15.105:6789/0 conn(0x7f8b80161470 legacy=0x7f8b8006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:07.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b6c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b6c004800 0x7f8b6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b6c0051c0 0x7f8b6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8006c170 con 0x7f8b6c003b40 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c00c0f0 con 0x7f8b6c0051c0 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c00c2f0 con 0x7f8b6c004800 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b6c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b6c004800 0x7f8b6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b6c0051c0 0x7f8b6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b6c004800 msgr2=0x7f8b6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b6c004800 0x7f8b6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b6c0051c0 msgr2=0x7f8b6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:07.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b6c0051c0 0x7f8b6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/231273138 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c003520 con 0x7f8b6c003b40 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b6c0051c0 0x7f8b6c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.991+0000 7f8b7ffff640 1 -- 172.21.15.105:0/231273138 reap_dead start 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.992+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b6c003f40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000c190 tx=0x7f8b6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a6b8df92ce3427 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.992+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70007cc0 con 0x7f8b6c003b40 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.992+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b70030070 con 0x7f8b6c003b40 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.992+0000 7f8b7effd640 1 -- 172.21.15.105:0/231273138 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70005d70 con 0x7f8b6c003b40 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.993+0000 7f8b85a64640 1 -- 172.21.15.105:0/231273138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 msgr2=0x7f8b6c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.993+0000 7f8b85a64640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b6c003f40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000c190 tx=0x7f8b6c005830 comp rx=0 tx=0).stop 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.993+0000 7f8b85a64640 1 -- 172.21.15.105:0/231273138 shutdown_connections 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.993+0000 7f8b85a64640 1 --2- 172.21.15.105:0/231273138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b6c003f40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.993+0000 7f8b85a64640 1 -- 172.21.15.105:0/231273138 >> 172.21.15.105:0/231273138 conn(0x7f8b800d3ce0 msgr2=0x7f8b800d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.993+0000 7f8b85a64640 1 -- 172.21.15.105:0/231273138 shutdown_connections 2024-03-08T08:29:07.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.993+0000 7f8b85a64640 1 -- 172.21.15.105:0/231273138 wait complete. 2024-03-08T08:29:08.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.994+0000 7f8b85a64640 1 Processor -- start 2024-03-08T08:29:08.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.994+0000 7f8b85a64640 1 -- start start 2024-03-08T08:29:08.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b85a64640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b800c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b85a64640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b800cbe00 0x7f8b801f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b85a64640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b801521d0 0x7f8b800cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b85a64640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8006d4f0 con 0x7f8b6c003b40 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b85a64640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8006d370 con 0x7f8b801521d0 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b85a64640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8006d670 con 0x7f8b800cbe00 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b800c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b800c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55278/0 (socket says 172.21.15.105:55278) 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/2952944361 learned_addr learned my addr 172.21.15.105:0/2952944361 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b801521d0 0x7f8b800cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b800cbe00 0x7f8b801f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b800cbe00 msgr2=0x7f8b801f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b800cbe00 0x7f8b801f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b801521d0 msgr2=0x7f8b800cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b801521d0 0x7f8b800cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/2952944361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80152660 con 0x7f8b6c003b40 2024-03-08T08:29:08.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.995+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b801521d0 0x7f8b800cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.996+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b800c9950 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8b700018f0 tx=0x7f8b700019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.996+0000 7f8b7cff9640 1 -- 172.21.15.105:0/2952944361 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70045070 con 0x7f8b6c003b40 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.996+0000 7f8b7cff9640 1 -- 172.21.15.105:0/2952944361 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7002c9c0 con 0x7f8b6c003b40 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.996+0000 7f8b7cff9640 1 -- 172.21.15.105:0/2952944361 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7002cd70 con 0x7f8b6c003b40 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.996+0000 7f8b85a64640 1 -- 172.21.15.105:0/2952944361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b800cc2c0 con 0x7f8b6c003b40 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.996+0000 7f8b85a64640 1 -- 172.21.15.105:0/2952944361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b800cc880 con 0x7f8b6c003b40 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.997+0000 7f8b7cff9640 1 -- 172.21.15.105:0/2952944361 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b70007070 con 0x7f8b6c003b40 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.998+0000 7f8b7cff9640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8b6006b500 0x7f8b6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.998+0000 7f8b7cff9640 1 -- 172.21.15.105:0/2952944361 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b700ab300 con 0x7f8b6c003b40 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.998+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8b6006b500 0x7f8b6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.998+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8b6006b500 0x7f8b6006d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8b800cecb0 tx=0x7f8b68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.999+0000 7f8b8863e740 1 Processor -- start 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:07.999+0000 7f8b8863e740 1 -- start start 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5bfd8a0 0x559ab5bfdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c018f0 0x559ab5c03390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b8863e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab5b67e30 con 0x559ab5c2d630 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b8863e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab5b10d20 con 0x559ab5bfd8a0 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b8863e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ab5b63ed0 con 0x559ab5c018f0 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55280/0 (socket says 172.21.15.105:55280) 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 -- 172.21.15.105:0/1522208921 learned_addr learned my addr 172.21.15.105:0/1522208921 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c018f0 0x559ab5c03390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5bfd8a0 0x559ab5bfdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 -- 172.21.15.105:0/1522208921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c018f0 msgr2=0x559ab5c03390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c018f0 0x559ab5c03390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 -- 172.21.15.105:0/1522208921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5bfd8a0 msgr2=0x559ab5bfdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5bfd8a0 0x559ab5bfdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 -- 172.21.15.105:0/1522208921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ab587c3f0 con 0x559ab5c2d630 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2dab0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8b8008df70 tx=0x7f8b78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a816bd643d5c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.000+0000 7f8b757fa640 1 -- 172.21.15.105:0/1522208921 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78017680 con 0x559ab5c2d630 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b757fa640 1 -- 172.21.15.105:0/1522208921 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7801d070 con 0x559ab5c2d630 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b757fa640 1 -- 172.21.15.105:0/1522208921 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78017a60 con 0x559ab5c2d630 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 -- 172.21.15.105:0/1522208921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 msgr2=0x559ab5c2dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2dab0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8b8008df70 tx=0x7f8b78009750 comp rx=0 tx=0).stop 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 -- 172.21.15.105:0/1522208921 shutdown_connections 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2dab0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c018f0 0x559ab5c03390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1522208921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5bfd8a0 0x559ab5bfdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 -- 172.21.15.105:0/1522208921 >> 172.21.15.105:0/1522208921 conn(0x559ab5bff920 msgr2=0x559ab5c011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 -- 172.21.15.105:0/1522208921 shutdown_connections 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.001+0000 7f8b8863e740 1 -- 172.21.15.105:0/1522208921 wait complete. 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.002+0000 7f8b8863e740 1 -- start start 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.002+0000 7f8b8863e740 1 Processor -- start 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c28a00 0x559ab5c30730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 0x559ab5c433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab5c01d80 con 0x559ab5c2d630 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab5b06390 con 0x559ab5c42f50 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ab5c02060 con 0x559ab5c28a00 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 10 client.0 _ll_register_callbacks cb 0x559ab5b23c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55296/0 (socket says 172.21.15.105:55296) 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/3627418230 learned_addr learned my addr 172.21.15.105:0/3627418230 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:08.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 0x559ab5c433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c28a00 0x559ab5c30730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/3627418230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c28a00 msgr2=0x559ab5c30730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c28a00 0x559ab5c30730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/3627418230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 msgr2=0x559ab5c433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 0x559ab5c433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/3627418230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ab59467d0 con 0x559ab5c2d630 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2e7a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8b70079a10 tx=0x7f8b70079640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59023e5e6091c4b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/3627418230 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7007dbd0 con 0x559ab5c2d630 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/3627418230 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b700760f0 con 0x559ab5c2d630 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 1 -- 172.21.15.105:0/3627418230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b7801d070 con 0x559ab5c2d630 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/3627418230 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b70082190 con 0x559ab5c2d630 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.003+0000 7f8b8863e740 10 client.5370 fetch_fsmap learned FSMap version 13 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b8863e740 1 -- 172.21.15.105:0/3627418230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x559ab5c2f660 con 0x559ab5c2d630 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/3627418230 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7801d070 con 0x559ab5c2d630 2024-03-08T08:29:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/3627418230 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b70078ed0 con 0x559ab5c2d630 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b8863e740 10 client.5370 fetch_fsmap finished waiting for FSMap version 13 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b8863e740 10 client.5370 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b8863e740 20 client.5370 populate_metadata read hostname 'smithi105' 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b8863e740 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x559ab5c28290 0x559ab5c65ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b8863e740 4 client.5370 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b8863e740 1 -- 172.21.15.105:0/3627418230 --> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ab5b75370 con 0x559ab5c28290 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x559ab5c28290 0x559ab5c65ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x559ab5c28290 0x559ab5c65ee0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeed21e7e8e845af server_cookie=c6d8b0375cd2715a in_seq=0 out_seq=0 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.004+0000 7f8b5e7fc640 10 client.5370 ms_handle_connect on v2:172.21.15.105:6836/2431299564 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/3627418230 <== mds.0 v2:172.21.15.105:6836/2431299564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559ab5b75370 con 0x559ab5c28290 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b5e7fc640 10 client.5370 handle_command_reply: tid=0 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 client.5370 shutdown 2024-03-08T08:29:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 -- 172.21.15.105:0/3627418230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 msgr2=0x559ab5c2e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2e7a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8b70079a10 tx=0x7f8b70079640 comp rx=0 tx=0).stop 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 -- 172.21.15.105:0/3627418230 shutdown_connections 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x559ab5c28290 0x559ab5c65ee0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 0x559ab5c433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b5e7fc640 0 client.5370 ms_handle_reset on v2:172.21.15.105:6836/2431299564 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c28a00 0x559ab5c30730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 --2- 172.21.15.105:0/3627418230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c2d630 0x559ab5c2e7a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 -- 172.21.15.105:0/3627418230 >> 172.21.15.105:0/3627418230 conn(0x559ab5bfd8a0 msgr2=0x559ab5c2ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 -- 172.21.15.105:0/3627418230 shutdown_connections 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 1 -- 172.21.15.105:0/3627418230 wait complete. 2024-03-08T08:29:08.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.005+0000 7f8b8863e740 20 client.5370 trim_cache size 0 max 16384 2024-03-08T08:29:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.016+0000 7f8b8863e740 1 Processor -- start 2024-03-08T08:29:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.017+0000 7f8b8863e740 1 -- start start 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.017+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c28a00 0x559ab5c2a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.017+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 0x559ab5c4e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.017+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.017+0000 7f8b8863e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b802006d0 con 0x559ab5c42f50 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.017+0000 7f8b8863e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801ff870 con 0x559ab5c28a00 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.017+0000 7f8b8863e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801f7a10 con 0x559ab5c2d630 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.017+0000 7f8b7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55300/0 (socket says 172.21.15.105:55300) 2024-03-08T08:29:08.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7ffff640 1 -- 172.21.15.105:0/2846708608 learned_addr learned my addr 172.21.15.105:0/2846708608 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c28a00 0x559ab5c2a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 0x559ab5c4e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7ffff640 1 -- 172.21.15.105:0/2846708608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 msgr2=0x559ab5c4e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 0x559ab5c4e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7ffff640 1 -- 172.21.15.105:0/2846708608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c28a00 msgr2=0x559ab5c2a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c28a00 0x559ab5c2a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7ffff640 1 -- 172.21.15.105:0/2846708608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ab5c43740 con 0x559ab5c42f50 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x559ab5c31d20 tx=0x7f8b7800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62e1c38c1c92222 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b74ff9640 1 -- 172.21.15.105:0/2846708608 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78003b90 con 0x559ab5c42f50 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b74ff9640 1 -- 172.21.15.105:0/2846708608 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78027070 con 0x559ab5c42f50 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.018+0000 7f8b74ff9640 1 -- 172.21.15.105:0/2846708608 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78017680 con 0x559ab5c42f50 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 -- 172.21.15.105:0/2846708608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c42f50 msgr2=0x559ab5bfc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x559ab5c31d20 tx=0x7f8b7800f000 comp rx=0 tx=0).stop 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 -- 172.21.15.105:0/2846708608 shutdown_connections 2024-03-08T08:29:08.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 0x559ab5c4e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 --2- 172.21.15.105:0/2846708608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c28a00 0x559ab5c2a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 -- 172.21.15.105:0/2846708608 >> 172.21.15.105:0/2846708608 conn(0x559ab5bfd8a0 msgr2=0x559ab5c30540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:08.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 -- 172.21.15.105:0/2846708608 shutdown_connections 2024-03-08T08:29:08.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 -- 172.21.15.105:0/2846708608 wait complete. 2024-03-08T08:29:08.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 -- start start 2024-03-08T08:29:08.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.019+0000 7f8b8863e740 1 Processor -- start 2024-03-08T08:29:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c28a00 0x559ab5c2ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 0x559ab5c9c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b8863e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b8863e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab5c04110 con 0x559ab5c28a00 2024-03-08T08:29:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b8863e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801ff3c0 con 0x559ab5c42f50 2024-03-08T08:29:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b8863e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ab5c46c80 con 0x559ab5c2d630 2024-03-08T08:29:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b8863e740 10 client.0 _ll_register_callbacks cb 0x7f8b70079850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c28a00 0x559ab5c2ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c28a00 0x559ab5c2ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55302/0 (socket says 172.21.15.105:55302) 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/1308106829 learned_addr learned my addr 172.21.15.105:0/1308106829 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7ffff640 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 0x559ab5c9c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/1308106829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 msgr2=0x559ab5c9c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 0x559ab5c9c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/1308106829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 msgr2=0x559ab5bfc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.020+0000 7f8b7f7fe640 1 -- 172.21.15.105:0/1308106829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ab5bfe120 con 0x559ab5c28a00 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b7f7fe640 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c28a00 0x559ab5c2ca20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8b70000f60 tx=0x7f8b700a9700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9f2a4484a49871 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1308106829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700795c0 con 0x559ab5c28a00 2024-03-08T08:29:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1308106829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b7007eb40 con 0x559ab5c28a00 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 1 -- 172.21.15.105:0/1308106829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b78027070 con 0x559ab5c28a00 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1308106829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b700a2050 con 0x559ab5c28a00 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 10 client.5376 fetch_fsmap learned FSMap version 13 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 1 -- 172.21.15.105:0/1308106829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x559ab5bfe120 con 0x559ab5c28a00 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1308106829 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78027070 con 0x559ab5c28a00 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1308106829 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b700757d0 con 0x559ab5c28a00 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 10 client.5376 fetch_fsmap finished waiting for FSMap version 13 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 10 client.5376 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 20 client.5376 populate_metadata read hostname 'smithi105' 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x559ab5bfcae0 0x559ab5c01f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 4 client.5376 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b8863e740 1 -- 172.21.15.105:0/1308106829 --> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] -- command(tid 0: {"prefix": "counter dump"}) v1 -- 0x559ab5c00080 con 0x559ab5bfcae0 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x559ab5bfcae0 0x559ab5c01f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b76ffd640 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x559ab5bfcae0 0x559ab5c01f90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99fab5e1cd8da2d1 server_cookie=7570bc1ca2dab672 in_seq=0 out_seq=0 2024-03-08T08:29:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.021+0000 7f8b5e7fc640 10 client.5376 ms_handle_connect on v2:172.21.15.105:6836/2431299564 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b5e7fc640 1 -- 172.21.15.105:0/1308106829 <== mds.0 v2:172.21.15.105:6836/2431299564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26939 (crc 0 0 0) 0x559ab5c00080 con 0x559ab5bfcae0 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b5e7fc640 10 client.5376 handle_command_reply: tid=0 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 client.5376 shutdown 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 -- 172.21.15.105:0/1308106829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c28a00 msgr2=0x559ab5c2ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c28a00 0x559ab5c2ca20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8b70000f60 tx=0x7f8b700a9700 comp rx=0 tx=0).stop 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 -- 172.21.15.105:0/1308106829 shutdown_connections 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.105:6836/2431299564,v1:172.21.15.105:6837/2431299564] conn(0x559ab5bfcae0 0x559ab5c01f90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559ab5c42f50 0x559ab5bfc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559ab5c2d630 0x559ab5c9c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 --2- 172.21.15.105:0/1308106829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559ab5c28a00 0x559ab5c2ca20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b5e7fc640 0 client.5376 ms_handle_reset on v2:172.21.15.105:6836/2431299564 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 -- 172.21.15.105:0/1308106829 >> 172.21.15.105:0/1308106829 conn(0x559ab5bfd8a0 msgr2=0x559ab5c41fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 -- 172.21.15.105:0/1308106829 shutdown_connections 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 1 -- 172.21.15.105:0/1308106829 wait complete. 2024-03-08T08:29:08.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.022+0000 7f8b8863e740 20 client.5376 trim_cache size 0 max 16384 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker": [ 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "0" 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:29:08.030 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "1" 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "2" 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.031 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_ready_timeouts": 0, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_connection_idle_timeouts": 0 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": [ 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 83, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 77, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 119089, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 215733, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 12, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.016078313, 2024-03-08T08:29:08.032 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.002824160, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.016587840, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000872447, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 77, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.090841649, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001179761 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000561, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000093 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 113104, 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 14848 2024-03-08T08:29:08.033 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": [ 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 3197, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 4237, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 866259, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1981603, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.242288546, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.160435716, 2024-03-08T08:29:08.034 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.114708751, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.001477714, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4236, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.067046181, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000015827 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6718, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000784665, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000116 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 99664 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.035 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": [ 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1342, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1342, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 342614, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 5292904, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.092026361, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.041056944, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.027507453, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.022016927, 2024-03-08T08:29:08.036 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1341, 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.181566154, 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000135396 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 462, 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000061151, 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000132 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.037 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": [ 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": [ 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-08T08:29:08.038 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2144, 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.641838681, 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000299365 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": [ 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-08T08:29:08.039 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000010937, 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000005468 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": [ 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 1206, 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 1206, 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1206, 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.353010477, 2024-03-08T08:29:08.040 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001121899 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 12, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 1011, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 1011, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 1011, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 102, 2024-03-08T08:29:08.041 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 104, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 2, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 1411, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 1409, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-08T08:29:08.042 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 1000, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 2890, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-08T08:29:08.043 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 174, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 1093, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 1185, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 1000, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 6, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 3 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": [ 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.044 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-08T08:29:08.045 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_client_metrics": [ 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "id": "5100" 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.046 INFO:teuthology.orchestra.run.smithi105.stdout: "num_clients": 2 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_client_metrics-cephfs": [ 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "client": "client.4853", 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": "0" 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_hits": 719, 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_miss": 74, 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_read_latency": 0.000000000, 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_write_latency": 0.000000000, 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T08:29:08.047 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_hits": 417, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_miss": 0, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_files": 83, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_inodes": 86, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "pinned_icaps": 86, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "total_inodes": 86, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_ops": 0, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_size": 0, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_ops": 0, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_size": 0 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": { 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "client": "client.5089", 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: "rank": "0" 2024-03-08T08:29:08.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_hits": 10032, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_miss": 19, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_read_latency": 0.000000000, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_write_latency": 0.000000000, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_metadata_latency": 0.000000000, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_hits": 1001, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_lease_miss": 0, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_files": 0, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "opened_inodes": 2, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "pinned_icaps": 2, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "total_inodes": 2, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_ops": 0, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "total_read_size": 0, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_ops": 1000, 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: "total_write_size": 2890 2024-03-08T08:29:08.049 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": [ 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 3013, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 3013, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 4, 2024-03-08T08:29:08.050 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 4, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 10994418, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1050, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.592275285, 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001516452 2024-03-08T08:29:08.051 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": [ 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 1014, 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1014, 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 13, 2024-03-08T08:29:08.052 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 13, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 0, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 1011, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 1011, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 104, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 1104, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 1000, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 69632, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215392 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": [ 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.053 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 1375, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 1206, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 1004, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1000, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.558526483, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000558526 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 103, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.655880427, 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.006367771 2024-03-08T08:29:08.054 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 101, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.135732254, 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001343883 2024-03-08T08:29:08.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.056 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.057 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002613698, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002613698 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.058 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.059 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.060 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000257615, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000257615 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.061 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.062 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.063 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": [ 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 2, 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 0, 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 1049, 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 524, 2024-03-08T08:29:08.064 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 22, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": [ 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T08:29:08.065 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-08T08:29:08.066 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 287619, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 1113, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 1408, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 16, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-08T08:29:08.067 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 5312, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 75, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 3095929, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 38405, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-08T08:29:08.068 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": [ 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 1101, 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 6893955, 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 31, 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 1101, 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1101, 2024-03-08T08:29:08.069 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.201957079, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001999961 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1101, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1137 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 1101, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 1101, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 11, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 1050, 2024-03-08T08:29:08.070 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 8, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 7, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 22, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-08T08:29:08.071 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-08T08:29:08.072 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 33, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 1, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 7, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 7, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 32, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 7 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.073 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": [ 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": [ 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.074 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": [ 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-03-08T08:29:08.075 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 4616, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 908092, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 4616, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4616, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 908092, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.076 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": [ 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:29:08.077 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 1101, 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 6893955, 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1101, 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 6893955, 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": [ 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:29:08.078 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 1101, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1101, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1101, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1101, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.079 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": [ 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:29:08.080 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x557e3b40e6e0": [ 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "labels": {}, 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "counters": { 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 3013, 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 6800114, 2024-03-08T08:29:08.081 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 3013, 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1050, 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 6800114, 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-08T08:29:08.082 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:29:08.084 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.084 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 -- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8b6006b500 msgr2=0x7f8b6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:08.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8b6006b500 0x7f8b6006d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8b800cecb0 tx=0x7f8b68009040 comp rx=0 tx=0).stop 2024-03-08T08:29:08.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 -- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 msgr2=0x7f8b800c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:08.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b800c9950 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8b700018f0 tx=0x7f8b700019c0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 -- 172.21.15.105:0/2952944361 shutdown_connections 2024-03-08T08:29:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f8b6006b500 0x7f8b6006d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b801521d0 0x7f8b800cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b800cbe00 0x7f8b801f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 --2- 172.21.15.105:0/2952944361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b6c003b40 0x7f8b800c9950 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 -- 172.21.15.105:0/2952944361 >> 172.21.15.105:0/2952944361 conn(0x7f8b800d3ce0 msgr2=0x7f8b800b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 -- 172.21.15.105:0/2952944361 shutdown_connections 2024-03-08T08:29:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:08.023+0000 7f8b85a64640 1 -- 172.21.15.105:0/2952944361 wait complete. 2024-03-08T08:29:08.085 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:08.086 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.087 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.094 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.149 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.150 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.151 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.201 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.206 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.262 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.262 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.267 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.315 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.321 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.375 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.376 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.427 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.434 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.487 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.488 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.493 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.541 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.547 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.601 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.601 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.655 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.661 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.714 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.715 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.721 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.770 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.774 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.827 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.836 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.887 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.893 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.901 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.952 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:08.987 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:29:08.989 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:29:09.005 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.007 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.009 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.068 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.086 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:09.120 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.121 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.126 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.184 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.233 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.234 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.243 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.272 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.289 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.300 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.308 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.337 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.354 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.403 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.403 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.408 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.410 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.419 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.467 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.483 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.483 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.519 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.522 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.522 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.548 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.548 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.584 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.613 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.613 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.632 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.639 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.677 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.679 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.696 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.743 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.744 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.746 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.751 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.808 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.810 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.858 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.859 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.860 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.866 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.924 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.924 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.931 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.974 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.976 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.040 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.040 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.086 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:10.087 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.089 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.096 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.151 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.152 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.153 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.203 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.206 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.264 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.264 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.269 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.317 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.323 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.377 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.378 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.382 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.429 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.436 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.490 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.490 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.495 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.543 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.548 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.603 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.603 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.608 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.657 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.663 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.716 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.717 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.723 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.772 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.776 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.829 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.830 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.838 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.889 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.895 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.903 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.905 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:10.954 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.007 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.007 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.011 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.070 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.087 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:11.122 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.123 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.128 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.186 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.235 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.236 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.245 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.274 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.291 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.302 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.310 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.339 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.355 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.356 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.404 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.405 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.410 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.410 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.421 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.469 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.485 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.485 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.521 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.549 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.550 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.586 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.735 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.735 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.735 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.736 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.736 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.736 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.736 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.737 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.746 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.748 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.750 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.753 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.810 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.812 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.860 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.861 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.862 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.868 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.926 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.926 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.933 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.975 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.978 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:11.981 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.041 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.042 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.088 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:12.089 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.091 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.098 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.153 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.153 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.154 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.266 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.266 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.270 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.319 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.325 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.379 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.380 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.384 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.431 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.438 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.492 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.492 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.497 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.545 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.551 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.605 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.605 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.610 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.659 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.664 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.718 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.719 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.725 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.774 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.778 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.831 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.832 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.840 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.891 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.897 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.905 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.907 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:12.956 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.009 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.009 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.072 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.088 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:13.123 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.125 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.130 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.188 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.237 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.237 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.247 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.276 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.293 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.304 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.312 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.342 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.357 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.358 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.405 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.407 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.412 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.412 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.423 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.471 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.487 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.488 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.523 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.526 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.526 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.551 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.552 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.588 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.617 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.617 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.636 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.639 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.643 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.681 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.683 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.700 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.747 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.748 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.755 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.812 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.814 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.862 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.863 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.864 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.870 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.928 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.928 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.936 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.977 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.978 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:13.983 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.043 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.043 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.090 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:14.091 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.093 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.100 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.155 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.155 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.156 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.208 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.211 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.268 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.269 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.272 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.321 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.327 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.381 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.382 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.386 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.433 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.440 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.493 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.499 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.547 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.552 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.607 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.607 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.612 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.661 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.666 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.720 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.720 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.726 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.776 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.780 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.833 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.834 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.842 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.893 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.899 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.907 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.909 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:14.958 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.011 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.011 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.015 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.074 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.091 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:15.125 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.127 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.132 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.190 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.239 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.239 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.249 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.278 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.295 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.306 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.314 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.359 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.360 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.406 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.409 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.414 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.414 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.425 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.473 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.489 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.490 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.525 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.528 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.528 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.553 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.589 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.617 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.619 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.638 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.641 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.645 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.683 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.685 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.702 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.749 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.750 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.752 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.754 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.757 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.814 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.816 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.864 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.865 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.866 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.872 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.930 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.930 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.938 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.980 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:15.985 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.045 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.046 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.091 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:16.093 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.095 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.102 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.157 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.158 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.158 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.210 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.213 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.270 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.271 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.274 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.323 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.329 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.381 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.384 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.388 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.435 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.442 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.495 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.496 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.501 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.549 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.554 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.609 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.610 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.614 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.663 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.668 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.722 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.722 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.728 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.779 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.782 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.836 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.836 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.844 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.895 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.901 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.909 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.911 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:16.960 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.013 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.014 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.017 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.076 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.092 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:17.125 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.129 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.134 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.194 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.241 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.241 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.251 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.297 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.308 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.316 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.344 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.362 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.408 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.411 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.416 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.416 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.428 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.475 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.491 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.492 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.527 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.530 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.530 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.555 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.556 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.591 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.619 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.621 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.640 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.643 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.647 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.685 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.687 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.704 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.751 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.752 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.754 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.756 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.759 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.816 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.818 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.866 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.867 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.868 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.873 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.932 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.932 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.940 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.982 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.982 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:17.987 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.047 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.048 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.092 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:18.095 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.098 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.104 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.159 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.160 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.160 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.212 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.218 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.272 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.273 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.276 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.325 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.332 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.383 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.386 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.390 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.437 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.444 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.497 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.498 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.503 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.552 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.556 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.611 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.612 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.616 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.665 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.670 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.724 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.724 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.730 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.781 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.784 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.838 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.838 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.846 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.898 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.904 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.911 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.913 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:18.962 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.016 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.016 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.019 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.078 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.093 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:19.128 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.131 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.136 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.194 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.243 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.243 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.253 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.282 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.299 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.311 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.318 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.346 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.363 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.364 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.410 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.413 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.418 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.418 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.429 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.477 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.493 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.494 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.529 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.557 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.558 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.593 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.621 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.623 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.642 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.646 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.649 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.687 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.690 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.707 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.754 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.756 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.759 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.761 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.818 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.820 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.868 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.869 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.870 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.875 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.934 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.934 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.941 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.984 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.984 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:19.990 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.048 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.050 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.093 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:20.097 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.099 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.106 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.161 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.162 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.162 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.214 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.217 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.278 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.327 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.333 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.385 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.388 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.392 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.439 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.446 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.499 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.500 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.505 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.554 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.558 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.614 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.614 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.618 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.667 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.672 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.726 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.726 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.732 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.783 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.786 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.840 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.840 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.848 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.898 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.906 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.913 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.915 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:20.964 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.018 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.018 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.021 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.080 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.094 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:21.130 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.134 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.138 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.196 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.245 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.246 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.256 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.285 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.301 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.313 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.320 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.348 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.366 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.366 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.412 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.416 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.420 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.420 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.432 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.479 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.496 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.496 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.532 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.534 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.535 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.559 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.560 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.595 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.623 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.625 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.645 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.648 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.651 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.689 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.692 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.755 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.756 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.759 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.761 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.763 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.820 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.822 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.870 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.871 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.872 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.877 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.936 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.937 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.943 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.986 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.986 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:21.990 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.050 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.052 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.094 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:22.099 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.101 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.108 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.163 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.164 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.164 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.216 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.219 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.277 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.277 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.280 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.329 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.336 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.388 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.391 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.394 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.441 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.449 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.501 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.502 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.507 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.556 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.560 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.616 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.616 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.621 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.728 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.735 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.785 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.788 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.841 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.842 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.851 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.901 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.908 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.915 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.917 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:22.966 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.020 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.023 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.083 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.095 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:23.132 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.136 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.140 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.198 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.247 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.248 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.258 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.287 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.303 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.315 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.323 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.351 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.368 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.368 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.414 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.418 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.422 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.423 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.434 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.481 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.498 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.498 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.534 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.536 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.537 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.561 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.562 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.597 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.624 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.627 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.647 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.650 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.692 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.694 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.758 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.758 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.761 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.763 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.765 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.823 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.825 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.872 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.873 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.874 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.879 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.938 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.939 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.945 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.988 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.988 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:23.992 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.052 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.054 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.096 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:24.101 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.103 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.110 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.166 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.166 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.167 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.218 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.221 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.279 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.279 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.281 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.331 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.336 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.390 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.392 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.396 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.443 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.451 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.503 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.504 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.510 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.558 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.562 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.618 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.623 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.672 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.676 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.730 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.730 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.737 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.787 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.791 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.842 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.844 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.853 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.903 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.910 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.917 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.919 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:24.968 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.022 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.022 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.025 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.085 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.097 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:25.134 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.137 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.142 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.200 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.249 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.249 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.260 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.289 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.305 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.315 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.325 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.352 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.370 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.370 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.416 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.420 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.425 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.425 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.436 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.482 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.500 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.500 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.536 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.539 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.539 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.599 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.626 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.629 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.649 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.651 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.656 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.694 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.696 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.713 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.759 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.759 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.763 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.765 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.766 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.825 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.827 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.873 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.874 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.876 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.881 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.941 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.941 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.947 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.990 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.991 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:25.994 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.054 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.056 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.097 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:26.103 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.105 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.112 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.168 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.168 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.168 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.220 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.224 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.281 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.281 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.283 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.333 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.338 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.392 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.395 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.398 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.446 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.505 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.506 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.512 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.560 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.564 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.620 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.620 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.625 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.674 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.679 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.732 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.732 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.739 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.789 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.793 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.844 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.846 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.855 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.904 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.912 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.919 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.921 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:26.970 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.024 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.024 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.027 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.087 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.098 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:27.136 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.139 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.144 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.202 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.251 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.251 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.262 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.291 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.307 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.317 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.327 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.354 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.372 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.372 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.418 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.422 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.427 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.427 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.438 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.483 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.502 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.502 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.538 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.541 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.541 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.566 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.566 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.601 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.628 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.631 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.651 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.653 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.656 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.696 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.698 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.715 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.761 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.762 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.765 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.767 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.768 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.827 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.829 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.875 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.875 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.878 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.883 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.943 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.943 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.949 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.992 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.992 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:27.996 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.056 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.058 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.100 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:28.105 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.107 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.114 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.170 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.171 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.171 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.222 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.226 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.283 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.283 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.285 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.335 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.340 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.394 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.397 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.400 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.448 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.455 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.507 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.508 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.514 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.562 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.566 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.622 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.622 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.627 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.676 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.681 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.734 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.735 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.741 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.791 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.795 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.846 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.848 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.857 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.906 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.914 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.921 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.923 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:28.972 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.026 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.026 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.029 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.089 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.100 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:29.138 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.141 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.204 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.253 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.253 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.264 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.293 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.309 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.319 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.329 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.356 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.374 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.374 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.421 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.424 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.429 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.429 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.440 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.485 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.504 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.504 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.540 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.543 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.543 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.568 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.568 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.604 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.630 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.633 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.653 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.655 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.658 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.698 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.700 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.717 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.762 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.764 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.767 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.769 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.770 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.829 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.877 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.877 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.880 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.885 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.945 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.945 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.951 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.994 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.994 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:29.998 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.059 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.060 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.101 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:30.107 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.108 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.116 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.172 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.172 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.173 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.224 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.228 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.285 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.285 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.287 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.337 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.342 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.396 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.399 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.402 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.450 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.457 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.510 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.510 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.516 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.564 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.569 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.624 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.625 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.629 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.678 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.683 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.736 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.737 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.743 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.793 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.797 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.848 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.850 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.859 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.908 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.923 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.925 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:30.974 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.028 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.029 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.031 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.091 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.101 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:31.140 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.143 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.148 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.206 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.255 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.266 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.295 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.312 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.321 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.358 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.376 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.376 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.423 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.426 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.431 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.431 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.442 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.487 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.506 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.506 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.542 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.545 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.545 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.569 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.570 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.606 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.632 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.635 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.655 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.657 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.660 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.700 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.702 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.718 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.764 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.766 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.768 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.771 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.772 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.831 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.833 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.879 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.879 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.882 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.887 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.947 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.947 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.954 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.996 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:31.996 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.000 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.061 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.061 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.103 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:32.109 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.110 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.119 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.173 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.173 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.173 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.226 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.230 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.287 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.287 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.289 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.339 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.344 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.398 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.401 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.404 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.452 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.459 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.512 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.512 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.518 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.566 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.570 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.627 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.627 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.631 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.680 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.685 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.738 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.739 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.745 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.796 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.799 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.850 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.852 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.861 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.910 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.917 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.926 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.928 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:32.976 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.030 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.031 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.033 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.093 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.104 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:33.142 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.145 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.150 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.208 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.257 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.257 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.268 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.296 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.314 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.322 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.333 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.359 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.378 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.379 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.425 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.428 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.433 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.444 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.489 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.508 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.508 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.544 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.547 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.547 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.572 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.572 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.608 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.634 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.638 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.657 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.659 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.662 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.702 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.704 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.720 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.766 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.768 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.770 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.773 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.774 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.833 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.881 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.881 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.884 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.889 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.949 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.949 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.998 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:33.998 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.002 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.063 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.063 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.104 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:34.112 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.112 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.121 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.175 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.175 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.175 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.228 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.232 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.289 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.289 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.291 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.341 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.346 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.400 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.403 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.406 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.454 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.461 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.514 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.514 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.520 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.568 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.572 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.630 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.630 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.633 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.682 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.687 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.741 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.741 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.747 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.798 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.801 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.852 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.854 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.863 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.912 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.919 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.928 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.930 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:34.977 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.032 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.033 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.035 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.095 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.105 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:35.145 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.147 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.153 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.211 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.259 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.259 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.269 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.298 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.316 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.324 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.335 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.361 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.380 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.381 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.427 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.430 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.435 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.435 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.446 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.492 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.510 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.510 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.546 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.549 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.549 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.573 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.574 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.610 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.637 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.640 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.659 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.661 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.664 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.704 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.706 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.722 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.768 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.770 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.772 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.775 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.776 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.835 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.837 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.883 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.883 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.886 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.950 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.951 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:35.958 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.000 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.000 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.004 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.065 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.065 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.105 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:36.114 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.114 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.123 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.177 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.177 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.177 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.230 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.234 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.291 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.291 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.293 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.343 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.349 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.402 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.405 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.408 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.456 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.463 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.516 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.516 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.522 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.570 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.574 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.631 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.631 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.635 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.684 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.689 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.743 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.743 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.749 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.800 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.803 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.855 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.856 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.865 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.914 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.922 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.930 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.932 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:36.979 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.033 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.034 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.037 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.098 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.106 DEBUG:tasks.cephfs.test_admin:jos debug hits 2024-03-08T08:29:37.106 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-08T08:29:37.106 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T08:29:37.147 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.149 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.150 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-08T08:29:37.154 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.193 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-08T08:29:37.282 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:29:37.283 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:37.284 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.284 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.285 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.285 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.301 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:29:37.301 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:37.301 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.318 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.327 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.337 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.358 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:29:37.363 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.383 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.383 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.429 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.432 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.437 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.437 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.449 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.494 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.504+0000 7f972fec9640 1 Processor -- start 2024-03-08T08:29:37.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.504+0000 7f972fec9640 1 -- start start 2024-03-08T08:29:37.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972fec9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f97280ccf70 0x7f972806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972fec9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97281457b0 con 0x7f97280ccb10 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972fec9640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97280b44e0 con 0x7f97281453d0 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972fec9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97280b47c0 con 0x7f97280ccf70 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972e466640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f97280ccf70 0x7f972806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972d464640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f97280ccb10 0x7f972806cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33466/0 (socket says 172.21.15.105:33466) 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972d464640 1 -- 172.21.15.105:0/1519081734 learned_addr learned my addr 172.21.15.105:0/1519081734 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972e466640 1 -- 172.21.15.105:0/1519081734 >> v1:172.21.15.144:6789/0 conn(0x7f97281453d0 legacy=0x7f97280cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972e466640 1 -- 172.21.15.105:0/1519081734 >> v1:172.21.15.105:6789/0 conn(0x7f97280ccb10 legacy=0x7f972806cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972e466640 1 -- 172.21.15.105:0/1519081734 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97280b4aa0 con 0x7f97280ccf70 2024-03-08T08:29:37.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.505+0000 7f972cc63640 1 -- 172.21.15.105:0/1519081734 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434188730 0 0) 0x7f97281457b0 con 0x7f97280ccb10 2024-03-08T08:29:37.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.506+0000 7f972e466640 1 --2- 172.21.15.105:0/1519081734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f972806d1d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9720009d90 tx=0x7f972002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af100ce7d143411 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:37.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.506+0000 7f972cc63640 1 -- 172.21.15.105:0/1519081734 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9720039070 con 0x7f97280ccf70 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.506+0000 7f972cc63640 1 -- 172.21.15.105:0/1519081734 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972002cc60 con 0x7f97280ccf70 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.506+0000 7f972cc63640 1 -- 172.21.15.105:0/1519081734 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9720034660 con 0x7f97280ccf70 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.507+0000 7f972fec9640 1 -- 172.21.15.105:0/1519081734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 msgr2=0x7f972806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.507+0000 7f972fec9640 1 --2- 172.21.15.105:0/1519081734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f972806d1d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9720009d90 tx=0x7f972002beb0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.507+0000 7f972fec9640 1 -- 172.21.15.105:0/1519081734 shutdown_connections 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.507+0000 7f972fec9640 1 --2- 172.21.15.105:0/1519081734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f972806d1d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.507+0000 7f972fec9640 1 -- 172.21.15.105:0/1519081734 >> 172.21.15.105:0/1519081734 conn(0x7f97280b1740 msgr2=0x7f97280b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.507+0000 7f972fec9640 1 -- 172.21.15.105:0/1519081734 shutdown_connections 2024-03-08T08:29:37.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.507+0000 7f972fec9640 1 -- 172.21.15.105:0/1519081734 wait complete. 2024-03-08T08:29:37.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.508+0000 7f972fec9640 1 Processor -- start 2024-03-08T08:29:37.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.508+0000 7f972fec9640 1 -- start start 2024-03-08T08:29:37.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972fec9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97280ccb10 0x7f9728141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972fec9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f97281e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972fec9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97281453d0 0x7f972815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972fec9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97280b4190 con 0x7f97280ccf70 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972fec9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97280cd3d0 con 0x7f97281453d0 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972fec9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97280cf510 con 0x7f97280ccb10 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f97281e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f97281e6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53656/0 (socket says 172.21.15.105:53656) 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 -- 172.21.15.105:0/789686370 learned_addr learned my addr 172.21.15.105:0/789686370 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972e466640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97281453d0 0x7f972815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 -- 172.21.15.105:0/789686370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97280ccb10 msgr2=0x7f9728141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97280ccb10 0x7f9728141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 -- 172.21.15.105:0/789686370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97281453d0 msgr2=0x7f972815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97281453d0 0x7f972815ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 -- 172.21.15.105:0/789686370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9728142190 con 0x7f97280ccf70 2024-03-08T08:29:37.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.509+0000 7f972d464640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f97281e6e00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f971800c3f0 tx=0x7f971800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:37.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.510+0000 7f971e7fc640 1 -- 172.21.15.105:0/789686370 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180074e0 con 0x7f97280ccf70 2024-03-08T08:29:37.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.510+0000 7f971e7fc640 1 -- 172.21.15.105:0/789686370 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97180047e0 con 0x7f97280ccf70 2024-03-08T08:29:37.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.510+0000 7f971e7fc640 1 -- 172.21.15.105:0/789686370 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718004c10 con 0x7f97280ccf70 2024-03-08T08:29:37.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.510+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9728142470 con 0x7f97280ccf70 2024-03-08T08:29:37.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.510+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9728142a30 con 0x7f97280ccf70 2024-03-08T08:29:37.513 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.514 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.511+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9728075120 con 0x7f97280ccf70 2024-03-08T08:29:37.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.511+0000 7f971e7fc640 1 -- 172.21.15.105:0/789686370 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9718011070 con 0x7f97280ccf70 2024-03-08T08:29:37.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.511+0000 7f971e7fc640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f970c06b4d0 0x7f970c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.512+0000 7f971e7fc640 1 -- 172.21.15.105:0/789686370 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9718089e00 con 0x7f97280ccf70 2024-03-08T08:29:37.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.512+0000 7f972dc65640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f970c06b4d0 0x7f970c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.512+0000 7f972dc65640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f970c06b4d0 0x7f970c06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9724003cc0 tx=0x7f97240079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:37.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.516+0000 7f971e7fc640 1 -- 172.21.15.105:0/789686370 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f971801a2b0 con 0x7f97280ccf70 2024-03-08T08:29:37.548 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.551 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.551 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.575 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.576 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.612 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.639 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.642 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.661 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.663 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.665 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.688+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97280af490 con 0x7f97280ccf70 2024-03-08T08:29:37.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.689+0000 7f971e7fc640 1 -- 172.21.15.105:0/789686370 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16681 (secure 0 0 0) 0x7f9718058c80 con 0x7f97280ccf70 2024-03-08T08:29:37.691 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:37.691 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":33,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:32.876978+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:28:29.680862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T08:28:30.179255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f970c06b4d0 msgr2=0x7f970c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f970c06b4d0 0x7f970c06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9724003cc0 tx=0x7f97240079b0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 msgr2=0x7f97281e6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f97281e6e00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f971800c3f0 tx=0x7f971800c8b0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 shutdown_connections 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f970c06b4d0 0x7f970c06d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97281453d0 0x7f972815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97280ccf70 0x7f97281e6e00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 --2- 172.21.15.105:0/789686370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97280ccb10 0x7f9728141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 >> 172.21.15.105:0/789686370 conn(0x7f97280b1740 msgr2=0x7f972815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:37.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 shutdown_connections 2024-03-08T08:29:37.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.691+0000 7f972fec9640 1 -- 172.21.15.105:0/789686370 wait complete. 2024-03-08T08:29:37.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:28:29.680862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:28:30.179255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:29:37.704 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:29:37.706 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.709 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.725 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.770 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.772 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.774 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.778 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.778 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.805+0000 7f573870c640 1 Processor -- start 2024-03-08T08:29:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.805+0000 7f573870c640 1 -- start start 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.806+0000 7f573870c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5730161470 0x7f573006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.806+0000 7f573870c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57300d48a0 con 0x7f5730161850 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.806+0000 7f573870c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57300d4b80 con 0x7f573006bcb0 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.806+0000 7f573870c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573006c170 con 0x7f5730161470 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.806+0000 7f5735ca7640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5730161850 0x7f573006b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33478/0 (socket says 172.21.15.105:33478) 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.806+0000 7f5735ca7640 1 -- 172.21.15.105:0/3343004747 learned_addr learned my addr 172.21.15.105:0/3343004747 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.806+0000 7f57364a8640 1 --2- 172.21.15.105:0/3343004747 >> v2:172.21.15.105:3300/0 conn(0x7f5730161470 0x7f573006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.807+0000 7f57364a8640 1 -- 172.21.15.105:0/3343004747 >> v1:172.21.15.144:6789/0 conn(0x7f573006bcb0 legacy=0x7f57300b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.807+0000 7f57364a8640 1 -- 172.21.15.105:0/3343004747 >> v1:172.21.15.105:6789/0 conn(0x7f5730161850 legacy=0x7f573006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.807+0000 7f57364a8640 1 -- 172.21.15.105:0/3343004747 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573006c450 con 0x7f5730161470 2024-03-08T08:29:37.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.807+0000 7f57354a6640 1 -- 172.21.15.105:0/3343004747 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123197741 0 0) 0x7f57300d48a0 con 0x7f5730161850 2024-03-08T08:29:37.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.807+0000 7f57364a8640 1 --2- 172.21.15.105:0/3343004747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161470 0x7f573006c830 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f572c009d90 tx=0x7f572c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b43147eb4e24dec server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:37.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f57354a6640 1 -- 172.21.15.105:0/3343004747 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c039070 con 0x7f5730161470 2024-03-08T08:29:37.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f57354a6640 1 -- 172.21.15.105:0/3343004747 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c02cc60 con 0x7f5730161470 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f57354a6640 1 -- 172.21.15.105:0/3343004747 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c034660 con 0x7f5730161470 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f573870c640 1 -- 172.21.15.105:0/3343004747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161470 msgr2=0x7f573006c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f573870c640 1 --2- 172.21.15.105:0/3343004747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161470 0x7f573006c830 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f572c009d90 tx=0x7f572c02beb0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f573870c640 1 -- 172.21.15.105:0/3343004747 shutdown_connections 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f573870c640 1 --2- 172.21.15.105:0/3343004747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161470 0x7f573006c830 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f573870c640 1 -- 172.21.15.105:0/3343004747 >> 172.21.15.105:0/3343004747 conn(0x7f57300d3ce0 msgr2=0x7f57300d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f573870c640 1 -- 172.21.15.105:0/3343004747 shutdown_connections 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.808+0000 7f573870c640 1 -- 172.21.15.105:0/3343004747 wait complete. 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.809+0000 7f573870c640 1 Processor -- start 2024-03-08T08:29:37.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.809+0000 7f573870c640 1 -- start start 2024-03-08T08:29:37.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f573870c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f573006bcb0 0x7f57300cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f573870c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730161470 0x7f57300c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f573870c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161850 0x7f57301f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f573870c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573006d4f0 con 0x7f5730161850 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f573870c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573006d370 con 0x7f5730161470 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f573870c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573006d670 con 0x7f573006bcb0 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5736ca9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161850 0x7f57301f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5736ca9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161850 0x7f57301f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53674/0 (socket says 172.21.15.105:53674) 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5736ca9640 1 -- 172.21.15.105:0/1481000453 learned_addr learned my addr 172.21.15.105:0/1481000453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5735ca7640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730161470 0x7f57300c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f57364a8640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f573006bcb0 0x7f57300cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5736ca9640 1 -- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f573006bcb0 msgr2=0x7f57300cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5736ca9640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f573006bcb0 0x7f57300cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5736ca9640 1 -- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730161470 msgr2=0x7f57300c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5736ca9640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730161470 0x7f57300c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5736ca9640 1 -- 172.21.15.105:0/1481000453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57300cb2f0 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.810+0000 7f5735ca7640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730161470 0x7f57300c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.811+0000 7f5736ca9640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161850 0x7f57301f63c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f572800c400 tx=0x7f572800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.811+0000 7f5726ffd640 1 -- 172.21.15.105:0/1481000453 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728017070 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.811+0000 7f5726ffd640 1 -- 172.21.15.105:0/1481000453 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57280052d0 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.811+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57301f5320 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.811+0000 7f5726ffd640 1 -- 172.21.15.105:0/1481000453 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728003810 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.811+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57301f5bc0 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.812+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57301f5710 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.812+0000 7f5726ffd640 1 -- 172.21.15.105:0/1481000453 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f572801a070 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.812+0000 7f5726ffd640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f571406b500 0x7f571406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.813+0000 7f5726ffd640 1 -- 172.21.15.105:0/1481000453 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5728089580 con 0x7f5730161850 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.813+0000 7f57364a8640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f571406b500 0x7f571406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:37.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.813+0000 7f57364a8640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f571406b500 0x7f571406d9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f572c036000 tx=0x7f572c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:37.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.817+0000 7f5726ffd640 1 -- 172.21.15.105:0/1481000453 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5728058400 con 0x7f5730161850 2024-03-08T08:29:37.837 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.839 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.885 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.885 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.888 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.952 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.953 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.960 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:37.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.988+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5730075120 con 0x7f5730161850 2024-03-08T08:29:37.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.989+0000 7f5726ffd640 1 -- 172.21.15.105:0/1481000453 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5728057d00 con 0x7f5730161850 2024-03-08T08:29:37.990 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.990+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f571406b500 msgr2=0x7f571406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.990+0000 7f573870c640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f571406b500 0x7f571406d9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f572c036000 tx=0x7f572c030000 comp rx=0 tx=0).stop 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.990+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161850 msgr2=0x7f57301f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.990+0000 7f573870c640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161850 0x7f57301f63c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f572800c400 tx=0x7f572800c8c0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.991+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 shutdown_connections 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.991+0000 7f573870c640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f571406b500 0x7f571406d9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.991+0000 7f573870c640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5730161850 0x7f57301f63c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.991+0000 7f573870c640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5730161470 0x7f57300c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.991+0000 7f573870c640 1 --2- 172.21.15.105:0/1481000453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f573006bcb0 0x7f57300cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.991+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 >> 172.21.15.105:0/1481000453 conn(0x7f57300d3ce0 msgr2=0x7f57300b8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:37.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.991+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 shutdown_connections 2024-03-08T08:29:37.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:37.991+0000 7f573870c640 1 -- 172.21.15.105:0/1481000453 wait complete. 2024-03-08T08:29:38.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:29:38.003 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.003 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.006 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.067 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.067 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.103+0000 7f18aec07640 1 Processor -- start 2024-03-08T08:29:38.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.103+0000 7f18aec07640 1 -- start start 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18aec07640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a81584b0 0x7f18a80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18aec07640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a815f170 con 0x7f18a80b2e80 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18aec07640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a80af8b0 con 0x7f18a815ece0 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18aec07640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80ad910 con 0x7f18a81584b0 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18ac9a3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f18a80b2e80 0x7f18a80b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33492/0 (socket says 172.21.15.105:33492) 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18ac9a3640 1 -- 172.21.15.105:0/814371199 learned_addr learned my addr 172.21.15.105:0/814371199 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667774544 0 0) 0x7f18a815f170 con 0x7f18a80b2e80 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1890003520 con 0x7f18a80b2e80 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18a7fff640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a81584b0 0x7f18a80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.104+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2547741598 0 0) 0x7f1890003520 con 0x7f18a80b2e80 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a81584b0 msgr2=0x7f18a80b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a81584b0 0x7f18a80b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 >> v1:172.21.15.144:6789/0 conn(0x7f18a815ece0 legacy=0x7f18a806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80adbf0 con 0x7f18a80b2e80 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f1898001670 con 0x7f18a80b2e80 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 >> v1:172.21.15.105:6789/0 conn(0x7f18a80b2e80 legacy=0x7f18a80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a81584b0 0x7f18900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1890004010 0x7f1890004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1890004d40 0x7f18900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80ad910 con 0x7f1890004010 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189000bb80 con 0x7f1890004d40 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189000bd80 con 0x7f18a81584b0 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a7fff640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1890004010 0x7f1890004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18ad1a4640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1890004d40 0x7f18900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18ac9a3640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a81584b0 0x7f18900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a7fff640 1 -- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a81584b0 msgr2=0x7f18900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:38.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a7fff640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a81584b0 0x7f18900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a7fff640 1 -- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1890004d40 msgr2=0x7f18900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a7fff640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1890004d40 0x7f18900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18a7fff640 1 -- 172.21.15.105:0/814371199 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890003520 con 0x7f1890004010 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.105+0000 7f18ad1a4640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1890004d40 0x7f18900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18a7fff640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1890004010 0x7f1890004800 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f189c00ba30 tx=0x7f1890005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae88558a3752f1c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c036020 con 0x7f1890004010 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c032640 con 0x7f1890004010 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18a77fe640 1 -- 172.21.15.105:0/814371199 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00b600 con 0x7f1890004010 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18aec07640 1 -- 172.21.15.105:0/814371199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1890004010 msgr2=0x7f1890004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18aec07640 1 --2- 172.21.15.105:0/814371199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1890004010 0x7f1890004800 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f189c00ba30 tx=0x7f1890005830 comp rx=0 tx=0).stop 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18ad1a4640 1 -- 172.21.15.105:0/814371199 reap_dead start 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18aec07640 1 -- 172.21.15.105:0/814371199 shutdown_connections 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18aec07640 1 -- 172.21.15.105:0/814371199 >> 172.21.15.105:0/814371199 conn(0x7f18a80cfdd0 msgr2=0x7f18a80d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.106+0000 7f18aec07640 1 -- 172.21.15.105:0/814371199 shutdown_connections 2024-03-08T08:29:38.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.107+0000 7f18aec07640 1 -- 172.21.15.105:0/814371199 wait complete. 2024-03-08T08:29:38.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.108+0000 7f18aec07640 1 Processor -- start 2024-03-08T08:29:38.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.108+0000 7f18aec07640 1 -- start start 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.108+0000 7f18aec07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81f7160 0x7f18a814fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.108+0000 7f18aec07640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a81f7540 0x7f18a81f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.108+0000 7f18aec07640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a80b1be0 0x7f18a80b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.108+0000 7f18aec07640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80ae490 con 0x7f18a81f7160 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.108+0000 7f18aec07640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80ae1b0 con 0x7f18a81f7540 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.108+0000 7f18aec07640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80ae9b0 con 0x7f18a80b1be0 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81f7160 0x7f18a814fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81f7160 0x7f18a814fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53700/0 (socket says 172.21.15.105:53700) 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 -- 172.21.15.105:0/3020011889 learned_addr learned my addr 172.21.15.105:0/3020011889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ad1a4640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a80b1be0 0x7f18a80b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18a7fff640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a81f7540 0x7f18a81f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 -- 172.21.15.105:0/3020011889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a80b1be0 msgr2=0x7f18a80b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a80b1be0 0x7f18a80b2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 -- 172.21.15.105:0/3020011889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a81f7540 msgr2=0x7f18a81f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a81f7540 0x7f18a81f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 -- 172.21.15.105:0/3020011889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8142a90 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18ac9a3640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81f7160 0x7f18a814fdf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1898007290 tx=0x7f1898007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18a57fa640 1 -- 172.21.15.105:0/3020011889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189800f960 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18a57fa640 1 -- 172.21.15.105:0/3020011889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898018070 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a8142d70 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18a57fa640 1 -- 172.21.15.105:0/3020011889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189800fd60 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.109+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a8143330 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.110+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a8075120 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.110+0000 7f18a57fa640 1 -- 172.21.15.105:0/3020011889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f189801d460 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.111+0000 7f18a57fa640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f189406b500 0x7f189406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.111+0000 7f18a57fa640 1 -- 172.21.15.105:0/3020011889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f189808a2e0 con 0x7f18a81f7160 2024-03-08T08:29:38.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.111+0000 7f18a7fff640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f189406b500 0x7f189406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.111+0000 7f18a7fff640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f189406b500 0x7f189406d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f189c00ba00 tx=0x7f189c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.116 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.116 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.115+0000 7f18a57fa640 1 -- 172.21.15.105:0/3020011889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1898059160 con 0x7f18a81f7160 2024-03-08T08:29:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.179 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.179 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.180 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.236 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.286+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18a8143500 con 0x7f18a81f7160 2024-03-08T08:29:38.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.286+0000 7f18a57fa640 1 -- 172.21.15.105:0/3020011889 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1898058a60 con 0x7f18a81f7160 2024-03-08T08:29:38.287 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:29:38.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f189406b500 msgr2=0x7f189406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f189406b500 0x7f189406d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f189c00ba00 tx=0x7f189c030000 comp rx=0 tx=0).stop 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81f7160 msgr2=0x7f18a814fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81f7160 0x7f18a814fdf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1898007290 tx=0x7f1898007750 comp rx=0 tx=0).stop 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 shutdown_connections 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f189406b500 0x7f189406d9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a80b1be0 0x7f18a80b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a81f7540 0x7f18a81f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 --2- 172.21.15.105:0/3020011889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81f7160 0x7f18a814fdf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 >> 172.21.15.105:0/3020011889 conn(0x7f18a80cfdd0 msgr2=0x7f18a8072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 shutdown_connections 2024-03-08T08:29:38.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.288+0000 7f18aec07640 1 -- 172.21.15.105:0/3020011889 wait complete. 2024-03-08T08:29:38.293 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.293 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.295 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.300 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:29:38.345 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.351 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.402+0000 7f1ae1255640 1 Processor -- start 2024-03-08T08:29:38.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.402+0000 7f1ae1255640 1 -- start start 2024-03-08T08:29:38.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.402+0000 7f1ae1255640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1adc0ce330 0x7f1adc0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.402+0000 7f1ae1255640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0cfb40 0x7f1adc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.402+0000 7f1ae1255640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1adc0b29e0 con 0x7f1adc161520 2024-03-08T08:29:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.402+0000 7f1ae1255640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1adc0d1930 con 0x7f1adc0ce330 2024-03-08T08:29:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.402+0000 7f1ae1255640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc06c770 con 0x7f1adc0cfb40 2024-03-08T08:29:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1ad259c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1adc0ce330 0x7f1adc0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1adb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0cfb40 0x7f1adc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1adad9d640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1adc161520 0x7f1adc0b2d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:48848/0 (socket says 172.21.15.105:48848) 2024-03-08T08:29:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1adad9d640 1 -- 172.21.15.105:0/2902938800 learned_addr learned my addr 172.21.15.105:0/2902938800 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:38.405 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1adb59e640 1 -- 172.21.15.105:0/2902938800 >> v2:172.21.15.144:3300/0 conn(0x7f1adc0ce330 msgr2=0x7f1adc0ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:38.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1adb59e640 1 --2- 172.21.15.105:0/2902938800 >> v2:172.21.15.144:3300/0 conn(0x7f1adc0ce330 0x7f1adc0ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1adb59e640 1 -- 172.21.15.105:0/2902938800 >> v1:172.21.15.144:6789/0 conn(0x7f1adc161520 legacy=0x7f1adc0b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1adb59e640 1 -- 172.21.15.105:0/2902938800 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0d1650 con 0x7f1adc0cfb40 2024-03-08T08:29:38.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.403+0000 7f1ad259c640 1 --2- 172.21.15.105:0/2902938800 >> v2:172.21.15.144:3300/0 conn(0x7f1adc0ce330 0x7f1adc0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.404+0000 7f1adb59e640 1 --2- 172.21.15.105:0/2902938800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0cfb40 0x7f1adc0cffd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009d90 tx=0x7f1ad402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68516251ccd7d34e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.404+0000 7f1ada59c640 1 -- 172.21.15.105:0/2902938800 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4039070 con 0x7f1adc0cfb40 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.404+0000 7f1ada59c640 1 -- 172.21.15.105:0/2902938800 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad402cc50 con 0x7f1adc0cfb40 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.404+0000 7f1ada59c640 1 -- 172.21.15.105:0/2902938800 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4034440 con 0x7f1adc0cfb40 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.405+0000 7f1ae1255640 1 -- 172.21.15.105:0/2902938800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0cfb40 msgr2=0x7f1adc0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.405+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2902938800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0cfb40 0x7f1adc0cffd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009d90 tx=0x7f1ad402bea0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.405+0000 7f1ae1255640 1 -- 172.21.15.105:0/2902938800 shutdown_connections 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.405+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2902938800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0cfb40 0x7f1adc0cffd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.405+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2902938800 >> v2:172.21.15.144:3300/0 conn(0x7f1adc0ce330 0x7f1adc0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.405+0000 7f1ae1255640 1 -- 172.21.15.105:0/2902938800 >> 172.21.15.105:0/2902938800 conn(0x7f1adc0b36a0 msgr2=0x7f1adc0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.405+0000 7f1ae1255640 1 -- 172.21.15.105:0/2902938800 shutdown_connections 2024-03-08T08:29:38.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.405+0000 7f1ae1255640 1 -- 172.21.15.105:0/2902938800 wait complete. 2024-03-08T08:29:38.408 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.406+0000 7f1ae1255640 1 Processor -- start 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.406+0000 7f1ae1255640 1 -- start start 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1ae1255640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0ce330 0x7f1adc154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1ae1255640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1adc0cfb40 0x7f1adc0b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1ae1255640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1adc161520 0x7f1adc0b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1ae1255640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc06ced0 con 0x7f1adc161520 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1ae1255640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc06ca40 con 0x7f1adc0cfb40 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1ae1255640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc06d050 con 0x7f1adc0ce330 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1adc161520 0x7f1adc0b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1adc161520 0x7f1adc0b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53708/0 (socket says 172.21.15.105:53708) 2024-03-08T08:29:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 -- 172.21.15.105:0/2093376906 learned_addr learned my addr 172.21.15.105:0/2093376906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adad9d640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0ce330 0x7f1adc154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1ad259c640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1adc0cfb40 0x7f1adc0b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 -- 172.21.15.105:0/2093376906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0ce330 msgr2=0x7f1adc154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0ce330 0x7f1adc154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 -- 172.21.15.105:0/2093376906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1adc0cfb40 msgr2=0x7f1adc0b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1adc0cfb40 0x7f1adc0b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 -- 172.21.15.105:0/2093376906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc155550 con 0x7f1adc161520 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.407+0000 7f1adb59e640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1adc161520 0x7f1adc0b6850 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009d90 tx=0x7f1ad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.408+0000 7f1ad3fff640 1 -- 172.21.15.105:0/2093376906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4039050 con 0x7f1adc161520 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.408+0000 7f1ad3fff640 1 -- 172.21.15.105:0/2093376906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad4045070 con 0x7f1adc161520 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.408+0000 7f1ad3fff640 1 -- 172.21.15.105:0/2093376906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4009450 con 0x7f1adc161520 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.408+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc0b4860 con 0x7f1adc161520 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.408+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc0b4e20 con 0x7f1adc161520 2024-03-08T08:29:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.409+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc075120 con 0x7f1adc161520 2024-03-08T08:29:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.409+0000 7f1ad3fff640 1 -- 172.21.15.105:0/2093376906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ad403e610 con 0x7f1adc161520 2024-03-08T08:29:38.412 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.409+0000 7f1ad3fff640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1abc06b4d0 0x7f1abc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.410+0000 7f1ad3fff640 1 -- 172.21.15.105:0/2093376906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ad40acc50 con 0x7f1adc161520 2024-03-08T08:29:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.410+0000 7f1adad9d640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1abc06b4d0 0x7f1abc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.410+0000 7f1adad9d640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1abc06b4d0 0x7f1abc06d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1acc000f60 tx=0x7f1acc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.414+0000 7f1ad3fff640 1 -- 172.21.15.105:0/2093376906 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ad40b2020 con 0x7f1adc161520 2024-03-08T08:29:38.458 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.465 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.517 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.518 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.524 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.573 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.576 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.584+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1adc0b79d0 con 0x7f1adc161520 2024-03-08T08:29:38.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.585+0000 7f1ad3fff640 1 -- 172.21.15.105:0/2093376906 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1adc155550 con 0x7f1adc161520 2024-03-08T08:29:38.586 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:29:38.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1abc06b4d0 msgr2=0x7f1abc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1abc06b4d0 0x7f1abc06d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1acc000f60 tx=0x7f1acc009040 comp rx=0 tx=0).stop 2024-03-08T08:29:38.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1adc161520 msgr2=0x7f1adc0b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1adc161520 0x7f1adc0b6850 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009d90 tx=0x7f1ad4004a40 comp rx=0 tx=0).stop 2024-03-08T08:29:38.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 shutdown_connections 2024-03-08T08:29:38.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f1abc06b4d0 0x7f1abc06d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1adc161520 0x7f1adc0b6850 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1adc0cfb40 0x7f1adc0b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 --2- 172.21.15.105:0/2093376906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1adc0ce330 0x7f1adc154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 >> 172.21.15.105:0/2093376906 conn(0x7f1adc0b36a0 msgr2=0x7f1adc0b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:38.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 shutdown_connections 2024-03-08T08:29:38.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.587+0000 7f1ae1255640 1 -- 172.21.15.105:0/2093376906 wait complete. 2024-03-08T08:29:38.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T08:29:38.633 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.633 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.687 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.691 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.698+0000 7f4620447640 1 Processor -- start 2024-03-08T08:29:38.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.698+0000 7f4620447640 1 -- start start 2024-03-08T08:29:38.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f4620447640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4618071dc0 0x7f46180b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f4620447640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461816ad20 con 0x7f461816a940 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f4620447640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180b3900 con 0x7f46180b49a0 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f4620447640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46180b3be0 con 0x7f4618071dc0 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d9e2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4618071dc0 0x7f46180b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461e1e3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f461816a940 0x7f46180716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33500/0 (socket says 172.21.15.105:33500) 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d9e2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4618071dc0 0x7f46180b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53722/0 (socket says 172.21.15.105:53722) 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461e1e3640 1 -- 172.21.15.105:0/794891671 learned_addr learned my addr 172.21.15.105:0/794891671 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d1e1640 1 -- 172.21.15.105:0/794891671 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595785908 0 0) 0x7f461816ad20 con 0x7f461816a940 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d1e1640 1 -- 172.21.15.105:0/794891671 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4604003500 con 0x7f461816a940 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d1e1640 1 -- 172.21.15.105:0/794891671 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470502902 0 0) 0x7f46180b3900 con 0x7f46180b49a0 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d1e1640 1 -- 172.21.15.105:0/794891671 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f461816ad20 con 0x7f46180b49a0 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d9e2640 1 -- 172.21.15.105:0/794891671 >> v1:172.21.15.144:6789/0 conn(0x7f46180b49a0 legacy=0x7f46180b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d1e1640 1 -- 172.21.15.105:0/794891671 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823857575 0 0) 0x7f4604003500 con 0x7f461816a940 2024-03-08T08:29:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d9e2640 1 -- 172.21.15.105:0/794891671 >> v1:172.21.15.105:6789/0 conn(0x7f461816a940 legacy=0x7f46180716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.699+0000 7f461d9e2640 1 -- 172.21.15.105:0/794891671 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180b3ec0 con 0x7f4618071dc0 2024-03-08T08:29:38.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.700+0000 7f461d9e2640 1 --2- 172.21.15.105:0/794891671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4618071dc0 0x7f46180b2b40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4608007810 tx=0x7f460802ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caddb2af1688942a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.700+0000 7f461d1e1640 1 -- 172.21.15.105:0/794891671 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608033070 con 0x7f4618071dc0 2024-03-08T08:29:38.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.700+0000 7f461d1e1640 1 -- 172.21.15.105:0/794891671 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460802f610 con 0x7f4618071dc0 2024-03-08T08:29:38.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.700+0000 7f461d1e1640 1 -- 172.21.15.105:0/794891671 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614001670 con 0x7f4618071dc0 2024-03-08T08:29:38.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.701+0000 7f4620447640 1 -- 172.21.15.105:0/794891671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4618071dc0 msgr2=0x7f46180b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.701+0000 7f4620447640 1 --2- 172.21.15.105:0/794891671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4618071dc0 0x7f46180b2b40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4608007810 tx=0x7f460802ba70 comp rx=0 tx=0).stop 2024-03-08T08:29:38.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.701+0000 7f4620447640 1 -- 172.21.15.105:0/794891671 shutdown_connections 2024-03-08T08:29:38.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.701+0000 7f4620447640 1 --2- 172.21.15.105:0/794891671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4618071dc0 0x7f46180b2b40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.701+0000 7f4620447640 1 -- 172.21.15.105:0/794891671 >> 172.21.15.105:0/794891671 conn(0x7f461806ac80 msgr2=0x7f461806b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:38.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.701+0000 7f4620447640 1 -- 172.21.15.105:0/794891671 shutdown_connections 2024-03-08T08:29:38.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.701+0000 7f4620447640 1 -- 172.21.15.105:0/794891671 wait complete. 2024-03-08T08:29:38.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.702+0000 7f4620447640 1 Processor -- start 2024-03-08T08:29:38.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.702+0000 7f4620447640 1 -- start start 2024-03-08T08:29:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f4620447640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4618071dc0 0x7f461815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f4620447640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46180b49a0 0x7f461815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f4620447640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461816a940 0x7f461816b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f4620447640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4618175490 con 0x7f461816a940 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f4620447640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180b4480 con 0x7f4618071dc0 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f4620447640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4618175610 con 0x7f46180b49a0 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461816a940 0x7f461816b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461816a940 0x7f461816b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53732/0 (socket says 172.21.15.105:53732) 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 -- 172.21.15.105:0/2452150560 learned_addr learned my addr 172.21.15.105:0/2452150560 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 -- 172.21.15.105:0/2452150560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46180b49a0 msgr2=0x7f461815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46180b49a0 0x7f461815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 -- 172.21.15.105:0/2452150560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4618071dc0 msgr2=0x7f461815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4618071dc0 0x7f461815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 -- 172.21.15.105:0/2452150560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461815cc30 con 0x7f461816a940 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f461e9e4640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461816a940 0x7f461816b150 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f461000c790 tx=0x7f461000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f460effd640 1 -- 172.21.15.105:0/2452150560 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610003b70 con 0x7f461816a940 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f460effd640 1 -- 172.21.15.105:0/2452150560 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610003d10 con 0x7f461816a940 2024-03-08T08:29:38.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461815e070 con 0x7f461816a940 2024-03-08T08:29:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f460effd640 1 -- 172.21.15.105:0/2452150560 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461000aa50 con 0x7f461816a940 2024-03-08T08:29:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.703+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461815e5f0 con 0x7f461816a940 2024-03-08T08:29:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.705+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4618075120 con 0x7f461816a940 2024-03-08T08:29:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.705+0000 7f460effd640 1 -- 172.21.15.105:0/2452150560 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f461000abf0 con 0x7f461816a940 2024-03-08T08:29:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.705+0000 7f460effd640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f45fc06b4d0 0x7f45fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.705+0000 7f460effd640 1 -- 172.21.15.105:0/2452150560 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f461008a140 con 0x7f461816a940 2024-03-08T08:29:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.705+0000 7f461e1e3640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f45fc06b4d0 0x7f45fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.706+0000 7f461e1e3640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f45fc06b4d0 0x7f45fc06d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4614005880 tx=0x7f4614000fc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.709+0000 7f460effd640 1 -- 172.21.15.105:0/2452150560 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4610059050 con 0x7f461816a940 2024-03-08T08:29:38.745 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.745 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.751 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.802 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.805 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.857 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.858 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.868 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.892+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f46180cefb0 con 0x7f461816a940 2024-03-08T08:29:38.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.892+0000 7f460effd640 1 -- 172.21.15.105:0/2452150560 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4610058950 con 0x7f461816a940 2024-03-08T08:29:38.893 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-08T08:29:38.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f45fc06b4d0 msgr2=0x7f45fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f45fc06b4d0 0x7f45fc06d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4614005880 tx=0x7f4614000fc0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461816a940 msgr2=0x7f461816b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461816a940 0x7f461816b150 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f461000c790 tx=0x7f461000cc50 comp rx=0 tx=0).stop 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 shutdown_connections 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f45fc06b4d0 0x7f45fc06d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f461816a940 0x7f461816b150 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46180b49a0 0x7f461815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 --2- 172.21.15.105:0/2452150560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4618071dc0 0x7f461815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 >> 172.21.15.105:0/2452150560 conn(0x7f461806ac80 msgr2=0x7f46180ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 shutdown_connections 2024-03-08T08:29:38.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:38.894+0000 7f4620447640 1 -- 172.21.15.105:0/2452150560 wait complete. 2024-03-08T08:29:38.906 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-08T08:29:38.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:38.906 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-08T08:29:38.916 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.924 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.932 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.934 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.943 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:29:38.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:38.943 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-08T08:29:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:38.998 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:39.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:38 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:39.035 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.036 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.039 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:29:39.054 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:29:39.100 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.149 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.151 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.157 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.215 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.264 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.273 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.320 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:39.328 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.333 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:39.339 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:39.365 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.385 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.385 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.431 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.435 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.439 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.439 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:39.448 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:39.451 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:39.496 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.514 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.514 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.550 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.553 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.554 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:39.563 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:39.577 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.578 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:39.614 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.641 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.644 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.663 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.665 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.667 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:39.677 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:39.708 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:39.711 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.727 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.772 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.774 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.777 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.780 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.780 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:39.789 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:39.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:39.839 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.841 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.887 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.887 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.890 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:39.895 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.901 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:39.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:39 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:39.954 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.955 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:39.962 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.004 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.005 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.018 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.069 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.069 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.118 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.118 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.127 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.131 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.181 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.181 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.181 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.234 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.238 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.248 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.295 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.295 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.297 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.347 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.352 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.360 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.406 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.409 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.460 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.467 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.473 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.519 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.521 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.527 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.575 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.579 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.588 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.635 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.635 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.639 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.689 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.693 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.701 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.743 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.746 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.754 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.804 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.807 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.817 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.859 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.860 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.870 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.918 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:40.926 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.932 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:40.934 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.934 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:40 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:40.936 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:40.983 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.038 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.038 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.041 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.052 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:41.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:41.102 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.151 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.154 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.159 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.173 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:41.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:41.217 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.266 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.266 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.275 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.289 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:41.305 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:41.330 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.342 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.367 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.385 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.387 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.399 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:41.433 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:41.441 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.441 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.453 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.498 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.511 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:41.516 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.516 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:41.555 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.555 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.579 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.580 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.616 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.629 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:41.641 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:41.646 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.667 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.669 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.711 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.713 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.729 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.741 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:41.774 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:41.776 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.782 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.782 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.841 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.844 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.856 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:41.889 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:41.892 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.897 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.956 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.957 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:41.964 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:41.970 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.005 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.010 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.071 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.072 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.083 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.120 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.129 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.183 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.184 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.184 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.197 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.240 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.297 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.297 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.299 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.313 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.354 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.408 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.411 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.414 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.426 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.469 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.521 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.523 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.529 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.541 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.581 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.637 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.637 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.641 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.653 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.695 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.748 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.749 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.756 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.769 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.809 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.861 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.862 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.871 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.884 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:42.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:42.928 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.936 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.936 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.985 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:42.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:42.998 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.040 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.043 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.104 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:43.118 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.153 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.156 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.219 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:43.231 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.267 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.268 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.307 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.332 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:43.344 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.344 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.369 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.389 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.435 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.443 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.443 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:43.455 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.456 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.518 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.518 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.557 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.558 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:43.571 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.581 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.582 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.644 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.648 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.669 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.671 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:43.685 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.713 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.715 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.777 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.778 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.784 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.784 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:43.797 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.844 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.891 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.894 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.898 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:43.911 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:43.952 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:43 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:43.958 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:43.966 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.007 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.012 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.025 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:44.073 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.122 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.131 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.143 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.185 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.185 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:44.242 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.254 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:44.298 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.299 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.356 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.369 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.410 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:44.413 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.471 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.483 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.523 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:44.525 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.583 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.596 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:44.639 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.639 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.697 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.710 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.823 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.863 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:44.864 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.930 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.934 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:44.938 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.938 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:44.943 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:44.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:44 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:45.042 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.045 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.155 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.158 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.270 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.270 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.309 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:45.322 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:45.346 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:45.371 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.391 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.437 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.445 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:45.457 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.458 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:45.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:45.520 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.520 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.560 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:45.572 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:45.583 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:45.584 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.646 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.650 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.671 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:45.684 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:45.717 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.779 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.780 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.786 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:45.800 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:45.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:45.846 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.893 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.896 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:45.909 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:45.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:45 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:45.959 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:45.968 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.009 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.021 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.075 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.124 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.137 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.187 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.187 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.199 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.300 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.300 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.313 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.410 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.415 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.423 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.525 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.527 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.538 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.641 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.642 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.654 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.752 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.753 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.758 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.766 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.865 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.867 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.878 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:46.940 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.940 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:46.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:46.953 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:46.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:46 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.047 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.059 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.157 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.160 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.171 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.272 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.272 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.283 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.348 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.360 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.373 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.393 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.405 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.440 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.459 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.470 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.522 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.522 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.534 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.585 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.586 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.596 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.648 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.652 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.659 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.717 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.719 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.730 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.781 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.782 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.792 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.848 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.895 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.906 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:47.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:47.961 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.970 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:47.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:47.981 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.090 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.189 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.201 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.302 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.313 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.417 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.430 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.529 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.543 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.643 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.656 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.754 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.767 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.867 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.879 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:48.942 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:48.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:48.954 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:48 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.046 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.058 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.159 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.172 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.274 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.286 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.375 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.388 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.442 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.452 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.524 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.536 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.588 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.601 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.654 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.666 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.719 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.730 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.784 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.796 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.850 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.862 INFO:tasks.cephfs.mount:Terminating background process 2024-03-08T08:29:49.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-03-08T08:29:49.963 INFO:teuthology.orchestra.run.smithi144.stderr:open_background: keeping file open 2024-03-08T08:29:49.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-08T08:29:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T08:29:49.978 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:49.978 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:50.036 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:50.036 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:50.094 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:29:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.205+0000 7fec44935640 1 Processor -- start 2024-03-08T08:29:50.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.205+0000 7fec44935640 1 -- start start 2024-03-08T08:29:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec44935640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec4006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec44935640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4015b670 con 0x7fec40072440 2024-03-08T08:29:50.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec44935640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4015b940 con 0x7fec400b4d00 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec44935640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4006ea50 con 0x7fec4006e0a0 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3e59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fec40072440 0x7fec40071020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56940/0 (socket says 172.21.15.105:56940) 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3e59c640 1 -- 172.21.15.105:0/3073621978 learned_addr learned my addr 172.21.15.105:0/3073621978 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3ed9d640 1 --2- 172.21.15.105:0/3073621978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec4006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3dd9b640 1 -- 172.21.15.105:0/3073621978 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107098662 0 0) 0x7fec4015b670 con 0x7fec40072440 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3dd9b640 1 -- 172.21.15.105:0/3073621978 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec2c003520 con 0x7fec40072440 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3dd9b640 1 -- 172.21.15.105:0/3073621978 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891334240 0 0) 0x7fec4015b940 con 0x7fec400b4d00 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3dd9b640 1 -- 172.21.15.105:0/3073621978 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec4015b670 con 0x7fec400b4d00 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3ed9d640 1 -- 172.21.15.105:0/3073621978 >> v1:172.21.15.144:6789/0 conn(0x7fec400b4d00 legacy=0x7fec4015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3ed9d640 1 -- 172.21.15.105:0/3073621978 >> v1:172.21.15.105:6789/0 conn(0x7fec40072440 legacy=0x7fec40071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.206+0000 7fec3ed9d640 1 -- 172.21.15.105:0/3073621978 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4015bc20 con 0x7fec4006e0a0 2024-03-08T08:29:50.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.207+0000 7fec3ed9d640 1 --2- 172.21.15.105:0/3073621978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec4006e510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec38009dc0 tx=0x7fec3802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a22ac13b413fbf96 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.207+0000 7fec3dd9b640 1 -- 172.21.15.105:0/3073621978 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38039070 con 0x7fec4006e0a0 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.207+0000 7fec3dd9b640 1 -- 172.21.15.105:0/3073621978 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3802ccd0 con 0x7fec4006e0a0 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.207+0000 7fec3dd9b640 1 -- 172.21.15.105:0/3073621978 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38034640 con 0x7fec4006e0a0 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.208+0000 7fec44935640 1 -- 172.21.15.105:0/3073621978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 msgr2=0x7fec4006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.208+0000 7fec44935640 1 --2- 172.21.15.105:0/3073621978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec4006e510 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec38009dc0 tx=0x7fec3802bf20 comp rx=0 tx=0).stop 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.208+0000 7fec44935640 1 -- 172.21.15.105:0/3073621978 shutdown_connections 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.208+0000 7fec44935640 1 --2- 172.21.15.105:0/3073621978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec4006e510 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.208+0000 7fec44935640 1 -- 172.21.15.105:0/3073621978 >> 172.21.15.105:0/3073621978 conn(0x7fec400b1740 msgr2=0x7fec400b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.208+0000 7fec44935640 1 -- 172.21.15.105:0/3073621978 shutdown_connections 2024-03-08T08:29:50.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.208+0000 7fec44935640 1 -- 172.21.15.105:0/3073621978 wait complete. 2024-03-08T08:29:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.209+0000 7fec44935640 1 Processor -- start 2024-03-08T08:29:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.209+0000 7fec44935640 1 -- start start 2024-03-08T08:29:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec44935640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec40154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec44935640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40072440 0x7fec4013d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec44935640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec400b4d00 0x7fec4013da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec44935640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec40072820 con 0x7fec400b4d00 2024-03-08T08:29:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec44935640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec40070b20 con 0x7fec40072440 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec44935640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec400729a0 con 0x7fec4006e0a0 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec3ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec400b4d00 0x7fec4013da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec3ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec400b4d00 0x7fec4013da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56400/0 (socket says 172.21.15.105:56400) 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec3ed9d640 1 -- 172.21.15.105:0/3132734965 learned_addr learned my addr 172.21.15.105:0/3132734965 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec3e59c640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec40154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.210+0000 7fec37fff640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40072440 0x7fec4013d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec3ed9d640 1 -- 172.21.15.105:0/3132734965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 msgr2=0x7fec40154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec3ed9d640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec40154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec3ed9d640 1 -- 172.21.15.105:0/3132734965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40072440 msgr2=0x7fec4013d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec3ed9d640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40072440 0x7fec4013d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec3ed9d640 1 -- 172.21.15.105:0/3132734965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec401524f0 con 0x7fec400b4d00 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec3ed9d640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec400b4d00 0x7fec4013da70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fec38009ee0 tx=0x7fec38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec36ffd640 1 -- 172.21.15.105:0/3132734965 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38039050 con 0x7fec400b4d00 2024-03-08T08:29:50.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec36ffd640 1 -- 172.21.15.105:0/3132734965 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38045070 con 0x7fec400b4d00 2024-03-08T08:29:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec40156c90 con 0x7fec400b4d00 2024-03-08T08:29:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec36ffd640 1 -- 172.21.15.105:0/3132734965 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38009450 con 0x7fec400b4d00 2024-03-08T08:29:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.211+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec40157250 con 0x7fec400b4d00 2024-03-08T08:29:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.212+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec40075120 con 0x7fec400b4d00 2024-03-08T08:29:50.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.212+0000 7fec36ffd640 1 -- 172.21.15.105:0/3132734965 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec3803e850 con 0x7fec400b4d00 2024-03-08T08:29:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.213+0000 7fec36ffd640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec2006b4d0 0x7fec2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.213+0000 7fec36ffd640 1 -- 172.21.15.105:0/3132734965 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec380ad2e0 con 0x7fec400b4d00 2024-03-08T08:29:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.213+0000 7fec3e59c640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec2006b4d0 0x7fec2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.213+0000 7fec3e59c640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec2006b4d0 0x7fec2006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fec30000f60 tx=0x7fec30007d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.217+0000 7fec36ffd640 1 -- 172.21.15.105:0/3132734965 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec3807bcf0 con 0x7fec400b4d00 2024-03-08T08:29:50.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.390+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec40151ac0 con 0x7fec400b4d00 2024-03-08T08:29:50.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.391+0000 7fec36ffd640 1 -- 172.21.15.105:0/3132734965 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16681 (secure 0 0 0) 0x7fec3807fd90 con 0x7fec400b4d00 2024-03-08T08:29:50.393 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:50.393 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":33,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:32.876978+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:28:29.680862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T08:28:30.179255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec2006b4d0 msgr2=0x7fec2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec2006b4d0 0x7fec2006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fec30000f60 tx=0x7fec30007d70 comp rx=0 tx=0).stop 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec400b4d00 msgr2=0x7fec4013da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec400b4d00 0x7fec4013da70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fec38009ee0 tx=0x7fec38004a40 comp rx=0 tx=0).stop 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 shutdown_connections 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fec2006b4d0 0x7fec2006d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec400b4d00 0x7fec4013da70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec40072440 0x7fec4013d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 --2- 172.21.15.105:0/3132734965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec4006e0a0 0x7fec40154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 >> 172.21.15.105:0/3132734965 conn(0x7fec400b1740 msgr2=0x7fec400b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 shutdown_connections 2024-03-08T08:29:50.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.393+0000 7fec44935640 1 -- 172.21.15.105:0/3132734965 wait complete. 2024-03-08T08:29:50.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:28:29.680862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:28:30.179255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:29:50.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:29:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.507+0000 7f2ca230d640 1 Processor -- start 2024-03-08T08:29:50.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.507+0000 7f2ca230d640 1 -- start start 2024-03-08T08:29:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2ca230d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c9c072440 0x7f2c9c071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2ca230d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c9c0b4d00 0x7f2c9c15af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2ca230d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c15b940 con 0x7f2c9c15b4a0 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2ca230d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c15bc20 con 0x7f2c9c072440 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2ca230d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c070840 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2c9bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c9c072440 0x7f2c9c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2c9bfff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2c9c072440 0x7f2c9c071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56404/0 (socket says 172.21.15.105:56404) 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2c9bfff640 1 -- 172.21.15.105:0/2003701920 learned_addr learned my addr 172.21.15.105:0/2003701920 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2c9b7fe640 1 --2- 172.21.15.105:0/2003701920 >> v2:172.21.15.144:3300/0 conn(0x7f2c9c0b4d00 0x7f2c9c15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2c9bfff640 1 -- 172.21.15.105:0/2003701920 >> v2:172.21.15.144:3300/0 conn(0x7f2c9c0b4d00 msgr2=0x7f2c9c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2c9bfff640 1 --2- 172.21.15.105:0/2003701920 >> v2:172.21.15.144:3300/0 conn(0x7f2c9c0b4d00 0x7f2c9c15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2c9bfff640 1 -- 172.21.15.105:0/2003701920 >> v1:172.21.15.144:6789/0 conn(0x7f2c9c15b4a0 legacy=0x7f2c9c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.508+0000 7f2c9bfff640 1 -- 172.21.15.105:0/2003701920 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c070b20 con 0x7f2c9c072440 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.509+0000 7f2c9bfff640 1 --2- 172.21.15.105:0/2003701920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c072440 0x7f2c9c071020 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0cf2b0 tx=0x7f2c9002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6db623b9ee8814 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.509+0000 7f2c9affd640 1 -- 172.21.15.105:0/2003701920 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90034070 con 0x7f2c9c072440 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.509+0000 7f2c9affd640 1 -- 172.21.15.105:0/2003701920 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c90030860 con 0x7f2c9c072440 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.509+0000 7f2c9affd640 1 -- 172.21.15.105:0/2003701920 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90030c30 con 0x7f2c9c072440 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.509+0000 7f2ca230d640 1 -- 172.21.15.105:0/2003701920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c072440 msgr2=0x7f2c9c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.509+0000 7f2ca230d640 1 --2- 172.21.15.105:0/2003701920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c072440 0x7f2c9c071020 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0cf2b0 tx=0x7f2c9002b9a0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.510+0000 7f2ca230d640 1 -- 172.21.15.105:0/2003701920 shutdown_connections 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.510+0000 7f2ca230d640 1 --2- 172.21.15.105:0/2003701920 >> v2:172.21.15.144:3300/0 conn(0x7f2c9c0b4d00 0x7f2c9c15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.510+0000 7f2ca230d640 1 --2- 172.21.15.105:0/2003701920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c072440 0x7f2c9c071020 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.510+0000 7f2ca230d640 1 -- 172.21.15.105:0/2003701920 >> 172.21.15.105:0/2003701920 conn(0x7f2c9c0b1740 msgr2=0x7f2c9c0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:50.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.510+0000 7f2ca230d640 1 -- 172.21.15.105:0/2003701920 shutdown_connections 2024-03-08T08:29:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.510+0000 7f2ca230d640 1 -- 172.21.15.105:0/2003701920 wait complete. 2024-03-08T08:29:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.511+0000 7f2ca230d640 1 Processor -- start 2024-03-08T08:29:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.511+0000 7f2ca230d640 1 -- start start 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca230d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c9c072440 0x7f2c9c150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca230d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c0b4d00 0x7f2c9c1666c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca230d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c9c15b4a0 0x7f2c9c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca230d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0729a0 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca230d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c072820 con 0x7f2c9c072440 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca230d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c06e7b0 con 0x7f2c9c15b4a0 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c0b4d00 0x7f2c9c1666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c0b4d00 0x7f2c9c1666c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56410/0 (socket says 172.21.15.105:56410) 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 -- 172.21.15.105:0/3339700947 learned_addr learned my addr 172.21.15.105:0/3339700947 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9bfff640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c9c072440 0x7f2c9c150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca08aa640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c9c15b4a0 0x7f2c9c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 -- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c9c15b4a0 msgr2=0x7f2c9c15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c9c15b4a0 0x7f2c9c15c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 -- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c9c072440 msgr2=0x7f2c9c150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c9c072440 0x7f2c9c150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 -- 172.21.15.105:0/3339700947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c14e300 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9bfff640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c9c072440 0x7f2c9c150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca08aa640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c9c15b4a0 0x7f2c9c15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c9b7fe640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c0b4d00 0x7f2c9c1666c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d800 tx=0x7f2c8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c98ff9640 1 -- 172.21.15.105:0/3339700947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00b6c0 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c98ff9640 1 -- 172.21.15.105:0/3339700947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c019070 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c15e6a0 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2c98ff9640 1 -- 172.21.15.105:0/3339700947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00baa0 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.512+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c15ec60 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.513+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c075120 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.514+0000 7f2c98ff9640 1 -- 172.21.15.105:0/3339700947 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c8c00bc40 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.514+0000 7f2c98ff9640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c8806b500 0x7f2c8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.514+0000 7f2c98ff9640 1 -- 172.21.15.105:0/3339700947 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c8c08a660 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.514+0000 7f2c9bfff640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c8806b500 0x7f2c8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.514+0000 7f2c9bfff640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c8806b500 0x7f2c8806d9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2c90036000 tx=0x7f2c9003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.518+0000 7f2c98ff9640 1 -- 172.21.15.105:0/3339700947 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c8c0594e0 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.687+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c9c06d000 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.687+0000 7f2c98ff9640 1 -- 172.21.15.105:0/3339700947 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c8c058de0 con 0x7f2c9c0b4d00 2024-03-08T08:29:50.688 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:29:50.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c8806b500 msgr2=0x7f2c8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c8806b500 0x7f2c8806d9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2c90036000 tx=0x7f2c9003b040 comp rx=0 tx=0).stop 2024-03-08T08:29:50.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c0b4d00 msgr2=0x7f2c9c1666c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c0b4d00 0x7f2c9c1666c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d800 tx=0x7f2c8c00dcc0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 shutdown_connections 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2c8806b500 0x7f2c8806d9c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c9c15b4a0 0x7f2c9c15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c9c0b4d00 0x7f2c9c1666c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 --2- 172.21.15.105:0/3339700947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c9c072440 0x7f2c9c150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 >> 172.21.15.105:0/3339700947 conn(0x7f2c9c0b1740 msgr2=0x7f2c9c0b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 shutdown_connections 2024-03-08T08:29:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.689+0000 7f2ca230d640 1 -- 172.21.15.105:0/3339700947 wait complete. 2024-03-08T08:29:50.701 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:29:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.802+0000 7f5f7db93640 1 Processor -- start 2024-03-08T08:29:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.802+0000 7f5f7db93640 1 -- start start 2024-03-08T08:29:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f7db93640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aad0 0x7f5f7815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f7db93640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f7815aeb0 0x7f5f7806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f7db93640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5f7806ec20 0x7f5f7806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f7db93640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7815ebb0 con 0x7f5f7815aeb0 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f7db93640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7815ee90 con 0x7f5f7806ec20 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f7db93640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7815f170 con 0x7f5f7815aad0 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f6effd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f7815aeb0 0x7f5f7806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f6effd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f7815aeb0 0x7f5f7806e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56426/0 (socket says 172.21.15.105:56426) 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.803+0000 7f5f6effd640 1 -- 172.21.15.105:0/893884458 learned_addr learned my addr 172.21.15.105:0/893884458 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f77fff640 1 --2- 172.21.15.105:0/893884458 >> v2:172.21.15.144:3300/0 conn(0x7f5f7806ec20 0x7f5f7806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f777fe640 1 --2- 172.21.15.105:0/893884458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aad0 0x7f5f7815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f6effd640 1 -- 172.21.15.105:0/893884458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aad0 msgr2=0x7f5f7815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f6effd640 1 --2- 172.21.15.105:0/893884458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aad0 0x7f5f7815e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f6effd640 1 -- 172.21.15.105:0/893884458 >> v2:172.21.15.144:3300/0 conn(0x7f5f7806ec20 msgr2=0x7f5f7806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f6effd640 1 --2- 172.21.15.105:0/893884458 >> v2:172.21.15.144:3300/0 conn(0x7f5f7806ec20 0x7f5f7806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f6effd640 1 -- 172.21.15.105:0/893884458 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7815f450 con 0x7f5f7815aeb0 2024-03-08T08:29:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f6effd640 1 --2- 172.21.15.105:0/893884458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aeb0 0x7f5f7806e6e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5f78067a40 tx=0x7f5f6002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a418530be1899955 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.804+0000 7f5f76ffd640 1 -- 172.21.15.105:0/893884458 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6002f8e0 con 0x7f5f7815aeb0 2024-03-08T08:29:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f76ffd640 1 -- 172.21.15.105:0/893884458 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6002fa80 con 0x7f5f7815aeb0 2024-03-08T08:29:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f76ffd640 1 -- 172.21.15.105:0/893884458 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6002fe50 con 0x7f5f7815aeb0 2024-03-08T08:29:50.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 -- 172.21.15.105:0/893884458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aeb0 msgr2=0x7f5f7806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 --2- 172.21.15.105:0/893884458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aeb0 0x7f5f7806e6e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5f78067a40 tx=0x7f5f6002db20 comp rx=0 tx=0).stop 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 -- 172.21.15.105:0/893884458 shutdown_connections 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 --2- 172.21.15.105:0/893884458 >> v2:172.21.15.144:3300/0 conn(0x7f5f7806ec20 0x7f5f7806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 --2- 172.21.15.105:0/893884458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aeb0 0x7f5f7806e6e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 --2- 172.21.15.105:0/893884458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aad0 0x7f5f7815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 -- 172.21.15.105:0/893884458 >> 172.21.15.105:0/893884458 conn(0x7f5f7806bd90 msgr2=0x7f5f7806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 -- 172.21.15.105:0/893884458 shutdown_connections 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.805+0000 7f5f7db93640 1 -- 172.21.15.105:0/893884458 wait complete. 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.806+0000 7f5f7db93640 1 Processor -- start 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.806+0000 7f5f7db93640 1 -- start start 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f7db93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f7806ec20 0x7f5f781515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f7db93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aad0 0x7f5f780b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f7db93640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aeb0 0x7f5f780b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f7db93640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f780728a0 con 0x7f5f7815aad0 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f7db93640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7815b330 con 0x7f5f7806ec20 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f7db93640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7806da00 con 0x7f5f7815aeb0 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f6effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aad0 0x7f5f780b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f6effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aad0 0x7f5f780b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56428/0 (socket says 172.21.15.105:56428) 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f6effd640 1 -- 172.21.15.105:0/721918349 learned_addr learned my addr 172.21.15.105:0/721918349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f77fff640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aeb0 0x7f5f780b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f777fe640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f7806ec20 0x7f5f781515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f6effd640 1 -- 172.21.15.105:0/721918349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aeb0 msgr2=0x7f5f780b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f6effd640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aeb0 0x7f5f780b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f6effd640 1 -- 172.21.15.105:0/721918349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f7806ec20 msgr2=0x7f5f781515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f6effd640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f7806ec20 0x7f5f781515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.807+0000 7f5f6effd640 1 -- 172.21.15.105:0/721918349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f781573d0 con 0x7f5f7815aad0 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.808+0000 7f5f6effd640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aad0 0x7f5f780b4250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5f60000c00 tx=0x7f5f600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.808+0000 7f5f74ff9640 1 -- 172.21.15.105:0/721918349 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60047070 con 0x7f5f7815aad0 2024-03-08T08:29:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.808+0000 7f5f74ff9640 1 -- 172.21.15.105:0/721918349 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60037c80 con 0x7f5f7815aad0 2024-03-08T08:29:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.808+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f781576b0 con 0x7f5f7815aad0 2024-03-08T08:29:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.808+0000 7f5f74ff9640 1 -- 172.21.15.105:0/721918349 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60040660 con 0x7f5f7815aad0 2024-03-08T08:29:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.808+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f78157c70 con 0x7f5f7815aad0 2024-03-08T08:29:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.809+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f78075120 con 0x7f5f7815aad0 2024-03-08T08:29:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.809+0000 7f5f74ff9640 1 -- 172.21.15.105:0/721918349 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f6004a070 con 0x7f5f7815aad0 2024-03-08T08:29:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.810+0000 7f5f74ff9640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f5806b4d0 0x7f5f5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.810+0000 7f5f74ff9640 1 -- 172.21.15.105:0/721918349 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f600ae500 con 0x7f5f7815aad0 2024-03-08T08:29:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.810+0000 7f5f777fe640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f5806b4d0 0x7f5f5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.810+0000 7f5f777fe640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f5806b4d0 0x7f5f5806d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f68006fd0 tx=0x7f5f68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.814+0000 7f5f74ff9640 1 -- 172.21.15.105:0/721918349 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f60051030 con 0x7f5f7815aad0 2024-03-08T08:29:50.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.988+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f780cd210 con 0x7f5f7815aad0 2024-03-08T08:29:50.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.988+0000 7f5f74ff9640 1 -- 172.21.15.105:0/721918349 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f6007d410 con 0x7f5f7815aad0 2024-03-08T08:29:50.989 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:29:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f5806b4d0 msgr2=0x7f5f5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f5806b4d0 0x7f5f5806d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f68006fd0 tx=0x7f5f68008040 comp rx=0 tx=0).stop 2024-03-08T08:29:50.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aad0 msgr2=0x7f5f780b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aad0 0x7f5f780b4250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5f60000c00 tx=0x7f5f600046c0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 shutdown_connections 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5f5806b4d0 0x7f5f5806d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f7815aeb0 0x7f5f780b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f7815aad0 0x7f5f780b4250 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 --2- 172.21.15.105:0/721918349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f7806ec20 0x7f5f781515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 >> 172.21.15.105:0/721918349 conn(0x7f5f7806bd90 msgr2=0x7f5f78154420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 shutdown_connections 2024-03-08T08:29:50.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:50.990+0000 7f5f7db93640 1 -- 172.21.15.105:0/721918349 wait complete. 2024-03-08T08:29:51.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:29:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.103+0000 7f5972e9d640 1 Processor -- start 2024-03-08T08:29:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.104+0000 7f5972e9d640 1 -- start start 2024-03-08T08:29:51.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.104+0000 7f5972e9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f596c06ce90 0x7f596c0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.104+0000 7f5972e9d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c0b2760 con 0x7f596c15dfc0 2024-03-08T08:29:51.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.104+0000 7f5972e9d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c0b2a40 con 0x7f596c0afc30 2024-03-08T08:29:51.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.104+0000 7f5972e9d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c0b3370 con 0x7f596c06ce90 2024-03-08T08:29:51.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.104+0000 7f5970c39640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f596c15dfc0 0x7f596c0b1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56954/0 (socket says 172.21.15.105:56954) 2024-03-08T08:29:51.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.104+0000 7f5970c39640 1 -- 172.21.15.105:0/1434040124 learned_addr learned my addr 172.21.15.105:0/1434040124 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256554466 0 0) 0x7f596c0b2760 con 0x7f596c15dfc0 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f595c003520 con 0x7f596c15dfc0 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f597143a640 1 --2- 172.21.15.105:0/1434040124 >> v2:172.21.15.144:3300/0 conn(0x7f596c06ce90 0x7f596c0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 985285309 0 0) 0x7f595c003520 con 0x7f596c15dfc0 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 >> v2:172.21.15.144:3300/0 conn(0x7f596c06ce90 msgr2=0x7f596c0b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 --2- 172.21.15.105:0/1434040124 >> v2:172.21.15.144:3300/0 conn(0x7f596c06ce90 0x7f596c0b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 >> v1:172.21.15.144:6789/0 conn(0x7f596c0afc30 legacy=0x7f596c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0b2d20 con 0x7f596c15dfc0 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f5960001670 con 0x7f596c15dfc0 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 >> v1:172.21.15.105:6789/0 conn(0x7f596c15dfc0 legacy=0x7f596c0b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f595c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f595c004800 0x7f595c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f595c0051c0 0x7f595c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0b3370 con 0x7f595c0039b0 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c00c060 con 0x7f595c0051c0 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c00c260 con 0x7f595c004800 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f5970c39640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f595c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f597143a640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f595c0051c0 0x7f595c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f596bfff640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f595c004800 0x7f595c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f5970c39640 1 -- 172.21.15.105:0/1434040124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f595c004800 msgr2=0x7f595c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f5970c39640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f595c004800 0x7f595c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f5970c39640 1 -- 172.21.15.105:0/1434040124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f595c0051c0 msgr2=0x7f595c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f5970c39640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f595c0051c0 0x7f595c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f597143a640 1 -- 172.21.15.105:0/1434040124 reap_dead start 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.105+0000 7f5970c39640 1 -- 172.21.15.105:0/1434040124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c003520 con 0x7f595c0039b0 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.106+0000 7f5970c39640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f595c003dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f596000c240 tx=0x7f595c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34a72342bf90840 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.106+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960007d10 con 0x7f595c0039b0 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.106+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5960030070 con 0x7f595c0039b0 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.106+0000 7f596b7fe640 1 -- 172.21.15.105:0/1434040124 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960005d90 con 0x7f595c0039b0 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.107+0000 7f5972e9d640 1 -- 172.21.15.105:0/1434040124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 msgr2=0x7f595c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.107+0000 7f5972e9d640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f595c003dd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f596000c240 tx=0x7f595c005830 comp rx=0 tx=0).stop 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.107+0000 7f5972e9d640 1 -- 172.21.15.105:0/1434040124 shutdown_connections 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.107+0000 7f5972e9d640 1 --2- 172.21.15.105:0/1434040124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f595c003dd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.107+0000 7f5972e9d640 1 -- 172.21.15.105:0/1434040124 >> 172.21.15.105:0/1434040124 conn(0x7f596c09aa00 msgr2=0x7f596c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.107+0000 7f5972e9d640 1 -- 172.21.15.105:0/1434040124 shutdown_connections 2024-03-08T08:29:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.107+0000 7f5972e9d640 1 -- 172.21.15.105:0/1434040124 wait complete. 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.108+0000 7f5972e9d640 1 Processor -- start 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.108+0000 7f5972e9d640 1 -- start start 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5972e9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f596c1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5972e9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f596c14a160 0x7f596c09f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5972e9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f596c15ffd0 0x7f596c0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5972e9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15e520 con 0x7f595c0039b0 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5972e9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15e3a0 con 0x7f596c15ffd0 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5972e9d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c06c780 con 0x7f596c14a160 2024-03-08T08:29:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f596c1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f596c1f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56446/0 (socket says 172.21.15.105:56446) 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 -- 172.21.15.105:0/1595458517 learned_addr learned my addr 172.21.15.105:0/1595458517 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f596bfff640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f596c14a160 0x7f596c09f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 -- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f596c14a160 msgr2=0x7f596c09f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f597143a640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f596c15ffd0 0x7f596c0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f596c14a160 0x7f596c09f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 -- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f596c15ffd0 msgr2=0x7f596c0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f596c15ffd0 0x7f596c0b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 -- 172.21.15.105:0/1595458517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c14a5e0 con 0x7f595c0039b0 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f597143a640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f596c15ffd0 0x7f596c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.109+0000 7f5970c39640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f596c1f4520 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f596002c520 tx=0x7f596c0b32e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.110+0000 7f59697fa640 1 -- 172.21.15.105:0/1595458517 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960045070 con 0x7f595c0039b0 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.110+0000 7f59697fa640 1 -- 172.21.15.105:0/1595458517 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5960000ed0 con 0x7f595c0039b0 2024-03-08T08:29:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.110+0000 7f59697fa640 1 -- 172.21.15.105:0/1595458517 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960007d10 con 0x7f595c0039b0 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.110+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c14a8b0 con 0x7f595c0039b0 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.110+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c14ae70 con 0x7f595c0039b0 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.111+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c075120 con 0x7f595c0039b0 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.111+0000 7f59697fa640 1 -- 172.21.15.105:0/1595458517 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5960004bf0 con 0x7f595c0039b0 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.111+0000 7f59697fa640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f595006b4d0 0x7f595006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.111+0000 7f59697fa640 1 -- 172.21.15.105:0/1595458517 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59600aad50 con 0x7f595c0039b0 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.112+0000 7f596bfff640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f595006b4d0 0x7f595006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.112+0000 7f596bfff640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f595006b4d0 0x7f595006d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5958007fd0 tx=0x7f5958009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.116+0000 7f59697fa640 1 -- 172.21.15.105:0/1595458517 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59600ae030 con 0x7f595c0039b0 2024-03-08T08:29:51.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.288+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f596c14b070 con 0x7f595c0039b0 2024-03-08T08:29:51.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.288+0000 7f59697fa640 1 -- 172.21.15.105:0/1595458517 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5960079580 con 0x7f595c0039b0 2024-03-08T08:29:51.290 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.290+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f595006b4d0 msgr2=0x7f595006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.290+0000 7f5972e9d640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f595006b4d0 0x7f595006d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5958007fd0 tx=0x7f5958009040 comp rx=0 tx=0).stop 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.290+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 msgr2=0x7f596c1f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.290+0000 7f5972e9d640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f596c1f4520 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f596002c520 tx=0x7f596c0b32e0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.291+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 shutdown_connections 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.291+0000 7f5972e9d640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f595006b4d0 0x7f595006d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.291+0000 7f5972e9d640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f596c15ffd0 0x7f596c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.291+0000 7f5972e9d640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f596c14a160 0x7f596c09f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.291+0000 7f5972e9d640 1 --2- 172.21.15.105:0/1595458517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f595c0039b0 0x7f596c1f4520 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.291+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 >> 172.21.15.105:0/1595458517 conn(0x7f596c09aa00 msgr2=0x7f596c06f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.291+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 shutdown_connections 2024-03-08T08:29:51.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.291+0000 7f5972e9d640 1 -- 172.21.15.105:0/1595458517 wait complete. 2024-03-08T08:29:51.302 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T08:29:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.402+0000 7fa20c748640 1 Processor -- start 2024-03-08T08:29:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.402+0000 7fa20c748640 1 -- start start 2024-03-08T08:29:51.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa20c748640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2040b4d00 0x7fa20415af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa20c748640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20415b940 con 0x7fa204072440 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa20c748640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20415bc20 con 0x7fa20415b4a0 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa20c748640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa204070840 con 0x7fa2040b4d00 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa20a4e4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa204072440 0x7fa204071020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56968/0 (socket says 172.21.15.105:56968) 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa20a4e4640 1 -- 172.21.15.105:0/3405583395 learned_addr learned my addr 172.21.15.105:0/3405583395 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859539185 0 0) 0x7fa20415b940 con 0x7fa204072440 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa209ce3640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2040b4d00 0x7fa20415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f0005730 con 0x7fa204072440 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1944385704 0 0) 0x7fa1f0005730 con 0x7fa204072440 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2040b4d00 msgr2=0x7fa20415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2040b4d00 0x7fa20415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 >> v1:172.21.15.144:6789/0 conn(0x7fa20415b4a0 legacy=0x7fa20406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204070b20 con 0x7fa204072440 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fa200001670 con 0x7fa204072440 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 >> v1:172.21.15.105:6789/0 conn(0x7fa204072440 legacy=0x7fa204071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2040b4d00 0x7fa1f0005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f0006170 0x7fa1f0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006f50 0x7fa1f00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa204070840 con 0x7fa1f0006170 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f000dcb0 con 0x7fa1f0006f50 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.403+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f000deb0 con 0x7fa2040b4d00 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa209ce3640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f0006170 0x7fa1f0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa20a4e4640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2040b4d00 0x7fa1f0005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa20ace5640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006f50 0x7fa1f00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa209ce3640 1 -- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2040b4d00 msgr2=0x7fa1f0005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa209ce3640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2040b4d00 0x7fa1f0005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa209ce3640 1 -- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006f50 msgr2=0x7fa1f00073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa209ce3640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006f50 0x7fa1f00073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa209ce3640 1 -- 172.21.15.105:0/3405583395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0005730 con 0x7fa1f0006170 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.404+0000 7fa209ce3640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f0006170 0x7fa1f0006a10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009ac0 tx=0x7fa1f0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c6665abfbd3bf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4039070 con 0x7fa1f0006170 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f402c6b0 con 0x7fa1f0006170 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa2094e2640 1 -- 172.21.15.105:0/3405583395 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4009690 con 0x7fa1f0006170 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa20c748640 1 -- 172.21.15.105:0/3405583395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f0006170 msgr2=0x7fa1f0006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa20c748640 1 --2- 172.21.15.105:0/3405583395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1f0006170 0x7fa1f0006a10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009ac0 tx=0x7fa1f0007a40 comp rx=0 tx=0).stop 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa20ace5640 1 -- 172.21.15.105:0/3405583395 reap_dead start 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa20c748640 1 -- 172.21.15.105:0/3405583395 shutdown_connections 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa20c748640 1 -- 172.21.15.105:0/3405583395 >> 172.21.15.105:0/3405583395 conn(0x7fa2040b1740 msgr2=0x7fa2040b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa20c748640 1 -- 172.21.15.105:0/3405583395 shutdown_connections 2024-03-08T08:29:51.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.405+0000 7fa20c748640 1 -- 172.21.15.105:0/3405583395 wait complete. 2024-03-08T08:29:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.406+0000 7fa20c748640 1 Processor -- start 2024-03-08T08:29:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.406+0000 7fa20c748640 1 -- start start 2024-03-08T08:29:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20c748640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006170 0x7fa20414dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20c748640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa204151ff0 0x7fa2040ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20c748640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2040ce7b0 0x7fa2041e53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20c748640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20406e930 con 0x7fa2040ce7b0 2024-03-08T08:29:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20c748640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20406e7b0 con 0x7fa1f0006170 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20c748640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20415af60 con 0x7fa204151ff0 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20ace5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2040ce7b0 0x7fa2041e53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20ace5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2040ce7b0 0x7fa2041e53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56460/0 (socket says 172.21.15.105:56460) 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20ace5640 1 -- 172.21.15.105:0/862291158 learned_addr learned my addr 172.21.15.105:0/862291158 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa209ce3640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa204151ff0 0x7fa2040ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20a4e4640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006170 0x7fa20414dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20ace5640 1 -- 172.21.15.105:0/862291158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa204151ff0 msgr2=0x7fa2040ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20ace5640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa204151ff0 0x7fa2040ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20ace5640 1 -- 172.21.15.105:0/862291158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006170 msgr2=0x7fa20414dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20ace5640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006170 0x7fa20414dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.407+0000 7fa20ace5640 1 -- 172.21.15.105:0/862291158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2041524b0 con 0x7fa2040ce7b0 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.408+0000 7fa20ace5640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2040ce7b0 0x7fa2041e53c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa2040cd570 tx=0x7fa1fc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.408+0000 7fa1faffd640 1 -- 172.21.15.105:0/862291158 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc003970 con 0x7fa2040ce7b0 2024-03-08T08:29:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.408+0000 7fa1faffd640 1 -- 172.21.15.105:0/862291158 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc018070 con 0x7fa2040ce7b0 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.408+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa204152790 con 0x7fa2040ce7b0 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.408+0000 7fa1faffd640 1 -- 172.21.15.105:0/862291158 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc003d50 con 0x7fa2040ce7b0 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.408+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa204152d50 con 0x7fa2040ce7b0 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.409+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa204075120 con 0x7fa2040ce7b0 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.409+0000 7fa1faffd640 1 -- 172.21.15.105:0/862291158 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1fc0050d0 con 0x7fa2040ce7b0 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.409+0000 7fa1faffd640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.410+0000 7fa1faffd640 1 -- 172.21.15.105:0/862291158 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1fc089ff0 con 0x7fa2040ce7b0 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.410+0000 7fa20a4e4640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.410+0000 7fa20a4e4640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa1e806b4d0 0x7fa1e806d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa200007cf0 tx=0x7fa200009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.414+0000 7fa1faffd640 1 -- 172.21.15.105:0/862291158 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1fc058de0 con 0x7fa2040ce7b0 2024-03-08T08:29:51.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.598+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa204071210 con 0x7fa2040ce7b0 2024-03-08T08:29:51.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.598+0000 7fa1faffd640 1 -- 172.21.15.105:0/862291158 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1fc0586e0 con 0x7fa2040ce7b0 2024-03-08T08:29:51.599 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa1e806b4d0 msgr2=0x7fa1e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa1e806b4d0 0x7fa1e806d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa200007cf0 tx=0x7fa200009040 comp rx=0 tx=0).stop 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2040ce7b0 msgr2=0x7fa2041e53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2040ce7b0 0x7fa2041e53c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa2040cd570 tx=0x7fa1fc00ec10 comp rx=0 tx=0).stop 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 shutdown_connections 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2040ce7b0 0x7fa2041e53c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa204151ff0 0x7fa2040ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 --2- 172.21.15.105:0/862291158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1f0006170 0x7fa20414dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 >> 172.21.15.105:0/862291158 conn(0x7fa2040b1740 msgr2=0x7fa20415a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 shutdown_connections 2024-03-08T08:29:51.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.600+0000 7fa20c748640 1 -- 172.21.15.105:0/862291158 wait complete. 2024-03-08T08:29:51.612 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-08T08:29:51.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:51.612 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-08T08:29:51.648 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:29:51.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:51.648 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-08T08:29:51.705 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:29:51.705 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:51.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16795 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T08:29:51.720 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:51.720 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:51.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1164 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T08:29:51.762 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:29:51.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.861+0000 7f6cb2ac9640 1 Processor -- start 2024-03-08T08:29:51.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.861+0000 7f6cb2ac9640 1 -- start start 2024-03-08T08:29:51.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb2ac9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6cac0b2c60 0x7f6cac0d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb2ac9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 0x7f6cac0cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb2ac9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac15e4e0 con 0x7f6cac0d0500 2024-03-08T08:29:51.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb2ac9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac15e770 con 0x7f6cac0b2c60 2024-03-08T08:29:51.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb2ac9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15ea50 con 0x7f6cac0cea70 2024-03-08T08:29:51.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb0865640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f6cac0d0500 0x7f6cac0d08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56974/0 (socket says 172.21.15.105:56974) 2024-03-08T08:29:51.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb0865640 1 -- 172.21.15.105:0/683356980 learned_addr learned my addr 172.21.15.105:0/683356980 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375175162 0 0) 0x7f6cac15e4e0 con 0x7f6cac0d0500 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb1066640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 0x7f6cac0cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c9c003520 con 0x7f6cac0d0500 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cabfff640 1 --2- 172.21.15.105:0/683356980 >> v2:172.21.15.144:3300/0 conn(0x7f6cac0b2c60 0x7f6cac0d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4233057187 0 0) 0x7f6c9c003520 con 0x7f6cac0d0500 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 msgr2=0x7f6cac0cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 0x7f6cac0cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 >> v2:172.21.15.144:3300/0 conn(0x7f6cac0b2c60 msgr2=0x7f6cac0d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 --2- 172.21.15.105:0/683356980 >> v2:172.21.15.144:3300/0 conn(0x7f6cac0b2c60 0x7f6cac0d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac15ed30 con 0x7f6cac0d0500 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f6ca0001670 con 0x7f6cac0d0500 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 >> v1:172.21.15.105:6789/0 conn(0x7f6cac0d0500 legacy=0x7f6cac0d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c0039b0 0x7f6c9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 0x7f6c9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c9c004d40 0x7f6c9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15e770 con 0x7f6c9c004d40 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15ea50 con 0x7f6c9c0039b0 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00b9c0 con 0x7f6cac0cea70 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb1066640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c9c004d40 0x7f6c9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.862+0000 7f6cb0865640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c0039b0 0x7f6c9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cabfff640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 0x7f6c9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cb1066640 1 -- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 msgr2=0x7f6c9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cb1066640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 0x7f6c9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cb1066640 1 -- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c0039b0 msgr2=0x7f6c9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cb1066640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c0039b0 0x7f6c9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cb1066640 1 -- 172.21.15.105:0/683356980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c003520 con 0x7f6c9c004d40 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cabfff640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0cea70 0x7f6c9c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cb0865640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c0039b0 0x7f6c9c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:29:51.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cb1066640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c9c004d40 0x7f6c9c0051c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400e170 tx=0x7f6c9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb439d011ae6a8bf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4039070 con 0x7f6c9c004d40 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca402cbf0 con 0x7f6c9c004d40 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.863+0000 7f6cab7fe640 1 -- 172.21.15.105:0/683356980 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4009c10 con 0x7f6c9c004d40 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.864+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/683356980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c9c004d40 msgr2=0x7f6c9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.864+0000 7f6cb2ac9640 1 --2- 172.21.15.105:0/683356980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c9c004d40 0x7f6c9c0051c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400e170 tx=0x7f6c9c005830 comp rx=0 tx=0).stop 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.864+0000 7f6cb1066640 1 -- 172.21.15.105:0/683356980 reap_dead start 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.864+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/683356980 shutdown_connections 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.864+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/683356980 >> 172.21.15.105:0/683356980 conn(0x7f6cac0d3c80 msgr2=0x7f6cac0d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.864+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/683356980 shutdown_connections 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.864+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/683356980 wait complete. 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.865+0000 7f6cb2ac9640 1 Processor -- start 2024-03-08T08:29:51.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.865+0000 7f6cb2ac9640 1 -- start start 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cb2ac9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c9c004d40 0x7f6cac0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cb2ac9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cac1f3ce0 0x7f6cac0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cb2ac9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0b85f0 0x7f6cac0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cb2ac9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0d4b10 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cb2ac9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15f530 con 0x7f6cac0b85f0 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cb2ac9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0d47a0 con 0x7f6c9c004d40 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cac1f3ce0 0x7f6cac0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cac1f3ce0 0x7f6cac0cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56474/0 (socket says 172.21.15.105:56474) 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cabfff640 1 -- 172.21.15.105:0/2304788127 learned_addr learned my addr 172.21.15.105:0/2304788127 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cabfff640 1 -- 172.21.15.105:0/2304788127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c9c004d40 msgr2=0x7f6cac0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cabfff640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c9c004d40 0x7f6cac0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cabfff640 1 -- 172.21.15.105:0/2304788127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0b85f0 msgr2=0x7f6cac0b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cb1066640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0b85f0 0x7f6cac0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cabfff640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0b85f0 0x7f6cac0b8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.866+0000 7f6cabfff640 1 -- 172.21.15.105:0/2304788127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0cdb40 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.867+0000 7f6cabfff640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cac1f3ce0 0x7f6cac0cd600 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c980116d0 tx=0x7f6c98011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.867+0000 7f6ca97fa640 1 -- 172.21.15.105:0/2304788127 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800f9c0 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.867+0000 7f6ca97fa640 1 -- 172.21.15.105:0/2304788127 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9800fb60 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.867+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac0cbd80 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.867+0000 7f6ca97fa640 1 -- 172.21.15.105:0/2304788127 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98015460 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.867+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac0cc340 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.868+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac075120 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.868+0000 7f6ca97fa640 1 -- 172.21.15.105:0/2304788127 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c98015600 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.868+0000 7f6ca97fa640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f6c9006b500 0x7f6c9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.869+0000 7f6ca97fa640 1 -- 172.21.15.105:0/2304788127 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c9808aba0 con 0x7f6cac1f3ce0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.869+0000 7f6cb0865640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f6c9006b500 0x7f6c9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:51.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.869+0000 7f6cb0865640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f6c9006b500 0x7f6c9006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0000f60 tx=0x7f6ca0001bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:51.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:51.873+0000 7f6ca97fa640 1 -- 172.21.15.105:0/2304788127 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c98019030 con 0x7f6cac1f3ce0 2024-03-08T08:29:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.084+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cac0d2040 con 0x7f6cac1f3ce0 2024-03-08T08:29:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.084+0000 7f6ca97fa640 1 -- 172.21.15.105:0/2304788127 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f6c98059a20 con 0x7f6cac1f3ce0 2024-03-08T08:29:52.085 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:52.086 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2759286099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2759286099},{"type":"v1","addr":"172.21.15.144:6834","nonce":2759286099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2788852991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2788852991},{"type":"v1","addr":"172.21.15.144:6835","nonce":2788852991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2226664928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2226664928},{"type":"v1","addr":"172.21.15.105:6835","nonce":2226664928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:28:31.203118+0000","modified":"2024-03-08T08:28:33.213155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2431299564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2431299564},{"type":"v1","addr":"172.21.15.105:6837","nonce":2431299564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T08:29:52.086 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 13 2024-03-08T08:29:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f6c9006b500 msgr2=0x7f6c9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f6c9006b500 0x7f6c9006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0000f60 tx=0x7f6ca0001bc0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cac1f3ce0 msgr2=0x7f6cac0cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cac1f3ce0 0x7f6cac0cd600 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c980116d0 tx=0x7f6c98011b90 comp rx=0 tx=0).stop 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 shutdown_connections 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f6c9006b500 0x7f6c9006d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac0b85f0 0x7f6cac0b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cac1f3ce0 0x7f6cac0cd600 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 --2- 172.21.15.105:0/2304788127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c9c004d40 0x7f6cac0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 >> 172.21.15.105:0/2304788127 conn(0x7f6cac0d3c80 msgr2=0x7f6cac0d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 shutdown_connections 2024-03-08T08:29:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.086+0000 7f6cb2ac9640 1 -- 172.21.15.105:0/2304788127 wait complete. 2024-03-08T08:29:52.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:29:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.196+0000 7f381caae640 1 Processor -- start 2024-03-08T08:29:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.197+0000 7f381caae640 1 -- start start 2024-03-08T08:29:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.197+0000 7f381caae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381806bcb0 0x7f381806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.197+0000 7f381caae640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38180d48a0 con 0x7f3818161470 2024-03-08T08:29:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.197+0000 7f381caae640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38180d4b80 con 0x7f3818161850 2024-03-08T08:29:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.197+0000 7f381caae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381806c170 con 0x7f381806bcb0 2024-03-08T08:29:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3818161470 0x7f381806b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56982/0 (socket says 172.21.15.105:56982) 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381659c640 1 -- 172.21.15.105:0/3391374689 learned_addr learned my addr 172.21.15.105:0/3391374689 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050793297 0 0) 0x7f38180d48a0 con 0x7f3818161470 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3804003520 con 0x7f3818161470 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f3815d9b640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381806bcb0 0x7f381806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2488363372 0 0) 0x7f3804003520 con 0x7f3818161470 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381806bcb0 msgr2=0x7f381806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381806bcb0 0x7f381806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 >> v1:172.21.15.144:6789/0 conn(0x7f3818161850 legacy=0x7f38180b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381806c450 con 0x7f3818161470 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f3808001670 con 0x7f3818161470 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 >> v1:172.21.15.105:6789/0 conn(0x7f3818161470 legacy=0x7f381806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38040039e0 0x7f3804003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804004800 0x7f3804004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381806bcb0 0x7f380400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381806c170 con 0x7f38040039e0 2024-03-08T08:29:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3804005550 con 0x7f3804004800 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380400bf00 con 0x7f381806bcb0 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381659c640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38040039e0 0x7f3804003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f3815d9b640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804004800 0x7f3804004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f3816d9d640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381806bcb0 0x7f380400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381659c640 1 -- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381806bcb0 msgr2=0x7f380400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.198+0000 7f381659c640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f381806bcb0 0x7f380400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.199+0000 7f381659c640 1 -- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804004800 msgr2=0x7f3804004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.199+0000 7f381659c640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804004800 0x7f3804004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.199+0000 7f381659c640 1 -- 172.21.15.105:0/3391374689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3804003520 con 0x7f38040039e0 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.199+0000 7f381659c640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38040039e0 0x7f3804003e00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f380800c240 tx=0x7f3804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd71798b1428aad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.199+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808007d10 con 0x7f38040039e0 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.199+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3808030070 con 0x7f38040039e0 2024-03-08T08:29:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.199+0000 7f381559a640 1 -- 172.21.15.105:0/3391374689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380802cd40 con 0x7f38040039e0 2024-03-08T08:29:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.200+0000 7f381caae640 1 -- 172.21.15.105:0/3391374689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38040039e0 msgr2=0x7f3804003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.200+0000 7f381caae640 1 --2- 172.21.15.105:0/3391374689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38040039e0 0x7f3804003e00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f380800c240 tx=0x7f3804005830 comp rx=0 tx=0).stop 2024-03-08T08:29:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.200+0000 7f3816d9d640 1 -- 172.21.15.105:0/3391374689 reap_dead start 2024-03-08T08:29:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.200+0000 7f381caae640 1 -- 172.21.15.105:0/3391374689 shutdown_connections 2024-03-08T08:29:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.200+0000 7f381caae640 1 -- 172.21.15.105:0/3391374689 >> 172.21.15.105:0/3391374689 conn(0x7f38180d3ce0 msgr2=0x7f38180d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.200+0000 7f381caae640 1 -- 172.21.15.105:0/3391374689 shutdown_connections 2024-03-08T08:29:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.200+0000 7f381caae640 1 -- 172.21.15.105:0/3391374689 wait complete. 2024-03-08T08:29:52.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.201+0000 7f381caae640 1 Processor -- start 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.201+0000 7f381caae640 1 -- start start 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f381caae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38040039e0 0x7f38181f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f381caae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38180a0660 0x7f38180a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f381caae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180c8480 0x7f38180c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f381caae640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381806d4f0 con 0x7f38180c8480 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f381caae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381806d370 con 0x7f38180a0660 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f381caae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381806d670 con 0x7f38040039e0 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3816d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180c8480 0x7f38180c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3816d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180c8480 0x7f38180c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56486/0 (socket says 172.21.15.105:56486) 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3816d9d640 1 -- 172.21.15.105:0/2789816721 learned_addr learned my addr 172.21.15.105:0/2789816721 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3815d9b640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38180a0660 0x7f38180a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f381659c640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38040039e0 0x7f38181f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3816d9d640 1 -- 172.21.15.105:0/2789816721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38040039e0 msgr2=0x7f38181f0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3816d9d640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38040039e0 0x7f38181f0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3816d9d640 1 -- 172.21.15.105:0/2789816721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38180a0660 msgr2=0x7f38180a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3816d9d640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38180a0660 0x7f38180a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.202+0000 7f3816d9d640 1 -- 172.21.15.105:0/2789816721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180c91b0 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.203+0000 7f3816d9d640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180c8480 0x7f38180c8900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3810000f80 tx=0x7f381000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.203+0000 7f380effd640 1 -- 172.21.15.105:0/2789816721 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810003970 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.203+0000 7f380effd640 1 -- 172.21.15.105:0/2789816721 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3810018070 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.203+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38180c9490 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.203+0000 7f380effd640 1 -- 172.21.15.105:0/2789816721 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810003d50 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.203+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38180c9a50 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.204+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3818075120 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.204+0000 7f380effd640 1 -- 172.21.15.105:0/2789816721 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38100050d0 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.205+0000 7f380effd640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.205+0000 7f381659c640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.205+0000 7f380effd640 1 -- 172.21.15.105:0/2789816721 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3810089e20 con 0x7f38180c8480 2024-03-08T08:29:52.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.205+0000 7f381659c640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37f806b4d0 0x7f37f806d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3808034000 tx=0x7f3808009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.209+0000 7f380effd640 1 -- 172.21.15.105:0/2789816721 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3810058d30 con 0x7f38180c8480 2024-03-08T08:29:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.380+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38180ce240 con 0x7f38180c8480 2024-03-08T08:29:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.381+0000 7f380effd640 1 -- 172.21.15.105:0/2789816721 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16681 (secure 0 0 0) 0x7f3810058630 con 0x7f38180c8480 2024-03-08T08:29:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:52.382 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":33,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:32.876978+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:28:29.680862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T08:28:30.179255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:29:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37f806b4d0 msgr2=0x7f37f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37f806b4d0 0x7f37f806d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3808034000 tx=0x7f3808009030 comp rx=0 tx=0).stop 2024-03-08T08:29:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180c8480 msgr2=0x7f38180c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180c8480 0x7f38180c8900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3810000f80 tx=0x7f381000ec10 comp rx=0 tx=0).stop 2024-03-08T08:29:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 shutdown_connections 2024-03-08T08:29:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180c8480 0x7f38180c8900 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38180a0660 0x7f38180a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 --2- 172.21.15.105:0/2789816721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38040039e0 0x7f38181f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 >> 172.21.15.105:0/2789816721 conn(0x7f38180d3ce0 msgr2=0x7f38180cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 shutdown_connections 2024-03-08T08:29:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.383+0000 7f381caae640 1 -- 172.21.15.105:0/2789816721 wait complete. 2024-03-08T08:29:52.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T08:27:27.135900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T08:28:29.680862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T08:28:30.179255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T08:29:52.395 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T08:29:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.497+0000 7f8527981640 1 Processor -- start 2024-03-08T08:29:52.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.497+0000 7f8527981640 1 -- start start 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f8527981640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f85200ccf70 0x7f852006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f8527981640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85201457b0 con 0x7f85201453d0 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f8527981640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85200b44e0 con 0x7f85200ccb10 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f8527981640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85200b47c0 con 0x7f85200ccf70 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f8525f1e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f85200ccf70 0x7f852006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f852571d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f85201453d0 0x7f85200cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56988/0 (socket says 172.21.15.105:56988) 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f852571d640 1 -- 172.21.15.105:0/1045794207 learned_addr learned my addr 172.21.15.105:0/1045794207 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f8525f1e640 1 -- 172.21.15.105:0/1045794207 >> v1:172.21.15.144:6789/0 conn(0x7f85200ccb10 legacy=0x7f852006cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f8525f1e640 1 -- 172.21.15.105:0/1045794207 >> v1:172.21.15.105:6789/0 conn(0x7f85201453d0 legacy=0x7f85200cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.498+0000 7f8525f1e640 1 -- 172.21.15.105:0/1045794207 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85200b4aa0 con 0x7f85200ccf70 2024-03-08T08:29:52.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8525f1e640 1 --2- 172.21.15.105:0/1045794207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85200ccf70 0x7f852006d1d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d90 tx=0x7f851802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a36177061a2069e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8517fff640 1 -- 172.21.15.105:0/1045794207 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518039070 con 0x7f85200ccf70 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8517fff640 1 -- 172.21.15.105:0/1045794207 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851802cc60 con 0x7f85200ccf70 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8517fff640 1 -- 172.21.15.105:0/1045794207 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518034660 con 0x7f85200ccf70 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8527981640 1 -- 172.21.15.105:0/1045794207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85200ccf70 msgr2=0x7f852006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8527981640 1 --2- 172.21.15.105:0/1045794207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85200ccf70 0x7f852006d1d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8518009d90 tx=0x7f851802beb0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8527981640 1 -- 172.21.15.105:0/1045794207 shutdown_connections 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8527981640 1 --2- 172.21.15.105:0/1045794207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85200ccf70 0x7f852006d1d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8527981640 1 -- 172.21.15.105:0/1045794207 >> 172.21.15.105:0/1045794207 conn(0x7f85200b1740 msgr2=0x7f85200b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8527981640 1 -- 172.21.15.105:0/1045794207 shutdown_connections 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.499+0000 7f8527981640 1 -- 172.21.15.105:0/1045794207 wait complete. 2024-03-08T08:29:52.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8527981640 1 Processor -- start 2024-03-08T08:29:52.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8527981640 1 -- start start 2024-03-08T08:29:52.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8527981640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85200ccb10 0x7f852013f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8527981640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85200ccf70 0x7f85200a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8527981640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85201453d0 0x7f85200a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8527981640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85200b4190 con 0x7f85201453d0 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8527981640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85200cd3d0 con 0x7f85200ccb10 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8527981640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85200cf510 con 0x7f85200ccf70 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8525f1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85201453d0 0x7f85200a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8525f1e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85201453d0 0x7f85200a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56504/0 (socket says 172.21.15.105:56504) 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.501+0000 7f8525f1e640 1 -- 172.21.15.105:0/4143540449 learned_addr learned my addr 172.21.15.105:0/4143540449 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8524f1c640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85200ccf70 0x7f85200a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f852571d640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85200ccb10 0x7f852013f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8525f1e640 1 -- 172.21.15.105:0/4143540449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85200ccf70 msgr2=0x7f85200a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8525f1e640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85200ccf70 0x7f85200a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8525f1e640 1 -- 172.21.15.105:0/4143540449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85200ccb10 msgr2=0x7f852013f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8525f1e640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85200ccb10 0x7f852013f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8525f1e640 1 -- 172.21.15.105:0/4143540449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852013fb40 con 0x7f85201453d0 2024-03-08T08:29:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8525f1e640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85201453d0 0x7f85200a99a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8518037040 tx=0x7f8518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8515ffb640 1 -- 172.21.15.105:0/4143540449 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518044030 con 0x7f85201453d0 2024-03-08T08:29:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8515ffb640 1 -- 172.21.15.105:0/4143540449 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851802ca60 con 0x7f85201453d0 2024-03-08T08:29:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852015c760 con 0x7f85201453d0 2024-03-08T08:29:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8515ffb640 1 -- 172.21.15.105:0/4143540449 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518039040 con 0x7f85201453d0 2024-03-08T08:29:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.502+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85200ac550 con 0x7f85201453d0 2024-03-08T08:29:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.503+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8520075120 con 0x7f85201453d0 2024-03-08T08:29:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.504+0000 7f8515ffb640 1 -- 172.21.15.105:0/4143540449 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8518047070 con 0x7f85201453d0 2024-03-08T08:29:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.504+0000 7f8515ffb640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f850406b4d0 0x7f850406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.504+0000 7f8515ffb640 1 -- 172.21.15.105:0/4143540449 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85180aca50 con 0x7f85201453d0 2024-03-08T08:29:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.504+0000 7f852571d640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f850406b4d0 0x7f850406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.504+0000 7f852571d640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f850406b4d0 0x7f850406d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8510000f60 tx=0x7f8510009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.508+0000 7f8515ffb640 1 -- 172.21.15.105:0/4143540449 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f851804e070 con 0x7f85201453d0 2024-03-08T08:29:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.677+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f852015e9f0 con 0x7f85201453d0 2024-03-08T08:29:52.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.677+0000 7f8515ffb640 1 -- 172.21.15.105:0/4143540449 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f851807b990 con 0x7f85201453d0 2024-03-08T08:29:52.678 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-08T08:29:52.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f850406b4d0 msgr2=0x7f850406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f850406b4d0 0x7f850406d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8510000f60 tx=0x7f8510009040 comp rx=0 tx=0).stop 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85201453d0 msgr2=0x7f85200a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85201453d0 0x7f85200a99a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8518037040 tx=0x7f8518004a40 comp rx=0 tx=0).stop 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 shutdown_connections 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f850406b4d0 0x7f850406d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85201453d0 0x7f85200a99a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85200ccf70 0x7f85200a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 --2- 172.21.15.105:0/4143540449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85200ccb10 0x7f852013f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 >> 172.21.15.105:0/4143540449 conn(0x7f85200b1740 msgr2=0x7f85200af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 shutdown_connections 2024-03-08T08:29:52.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.679+0000 7f8527981640 1 -- 172.21.15.105:0/4143540449 wait complete. 2024-03-08T08:29:52.691 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T08:29:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.792+0000 7f143e948640 1 Processor -- start 2024-03-08T08:29:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.792+0000 7f143e948640 1 -- start start 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.793+0000 7f143e948640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1438161470 0x7f143806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.793+0000 7f143e948640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 0x7f14380d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.793+0000 7f143e948640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380d48a0 con 0x7f1438161850 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.793+0000 7f143e948640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14380d2ba0 con 0x7f1438161470 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.793+0000 7f143e948640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14380d4b80 con 0x7f143806bcb0 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.793+0000 7f1437fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1438161470 0x7f143806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.793+0000 7f1437fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1438161470 0x7f143806c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56508/0 (socket says 172.21.15.105:56508) 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.793+0000 7f1437fff640 1 -- 172.21.15.105:0/688428938 learned_addr learned my addr 172.21.15.105:0/688428938 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.794+0000 7f1437fff640 1 -- 172.21.15.105:0/688428938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 msgr2=0x7f14380d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.794+0000 7f1437fff640 1 --2- 172.21.15.105:0/688428938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 0x7f14380d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.794+0000 7f1437fff640 1 -- 172.21.15.105:0/688428938 >> v1:172.21.15.144:6789/0 conn(0x7f1438161850 legacy=0x7f143806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:52.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.794+0000 7f1437fff640 1 -- 172.21.15.105:0/688428938 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143806c170 con 0x7f1438161470 2024-03-08T08:29:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.794+0000 7f1437fff640 1 --2- 172.21.15.105:0/688428938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161470 0x7f143806c830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f143808c850 tx=0x7f142802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8534646450be2ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.794+0000 7f1436ffd640 1 -- 172.21.15.105:0/688428938 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14280317c0 con 0x7f1438161470 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f1436ffd640 1 -- 172.21.15.105:0/688428938 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428037070 con 0x7f1438161470 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f1436ffd640 1 -- 172.21.15.105:0/688428938 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428031b80 con 0x7f1438161470 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f143e948640 1 -- 172.21.15.105:0/688428938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161470 msgr2=0x7f143806c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f143e948640 1 --2- 172.21.15.105:0/688428938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161470 0x7f143806c830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f143808c850 tx=0x7f142802b860 comp rx=0 tx=0).stop 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f143e948640 1 -- 172.21.15.105:0/688428938 shutdown_connections 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f143e948640 1 --2- 172.21.15.105:0/688428938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 0x7f14380d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f143e948640 1 --2- 172.21.15.105:0/688428938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161470 0x7f143806c830 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f143e948640 1 -- 172.21.15.105:0/688428938 >> 172.21.15.105:0/688428938 conn(0x7f14380d3ce0 msgr2=0x7f14380d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f143e948640 1 -- 172.21.15.105:0/688428938 shutdown_connections 2024-03-08T08:29:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.795+0000 7f143e948640 1 -- 172.21.15.105:0/688428938 wait complete. 2024-03-08T08:29:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.796+0000 7f143e948640 1 Processor -- start 2024-03-08T08:29:52.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.796+0000 7f143e948640 1 -- start start 2024-03-08T08:29:52.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143e948640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 0x7f143815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143e948640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1438161470 0x7f14381f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143e948640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161850 0x7f14380a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143e948640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143806cf40 con 0x7f1438161850 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143e948640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143806d370 con 0x7f1438161470 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143e948640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143809de70 con 0x7f143806bcb0 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143cee5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161850 0x7f14380a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143cee5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161850 0x7f14380a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56520/0 (socket says 172.21.15.105:56520) 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.797+0000 7f143cee5640 1 -- 172.21.15.105:0/2416308136 learned_addr learned my addr 172.21.15.105:0/2416308136 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f1437fff640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 0x7f143815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f14377fe640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1438161470 0x7f14381f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f143cee5640 1 -- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 msgr2=0x7f143815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f143cee5640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 0x7f143815e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f143cee5640 1 -- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1438161470 msgr2=0x7f14381f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f143cee5640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1438161470 0x7f14381f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f143cee5640 1 -- 172.21.15.105:0/2416308136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143809e150 con 0x7f1438161850 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f1437fff640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 0x7f143815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f14377fe640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1438161470 0x7f14381f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f143cee5640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161850 0x7f14380a12c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f143000b9e0 tx=0x7f143000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f1434ff9640 1 -- 172.21.15.105:0/2416308136 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430019070 con 0x7f1438161850 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f1434ff9640 1 -- 172.21.15.105:0/2416308136 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143000cb40 con 0x7f1438161850 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143809e430 con 0x7f1438161850 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f1434ff9640 1 -- 172.21.15.105:0/2416308136 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430014440 con 0x7f1438161850 2024-03-08T08:29:52.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.798+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143809e9f0 con 0x7f1438161850 2024-03-08T08:29:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.799+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143809ebf0 con 0x7f1438161850 2024-03-08T08:29:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.799+0000 7f1434ff9640 1 -- 172.21.15.105:0/2416308136 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f143001c070 con 0x7f1438161850 2024-03-08T08:29:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.800+0000 7f1434ff9640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f142406b4d0 0x7f142406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.800+0000 7f1434ff9640 1 -- 172.21.15.105:0/2416308136 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1430089e60 con 0x7f1438161850 2024-03-08T08:29:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.800+0000 7f1437fff640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f142406b4d0 0x7f142406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:52.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.800+0000 7f1437fff640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f142406b4d0 0x7f142406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1428036000 tx=0x7f1428008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:52.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.804+0000 7f1434ff9640 1 -- 172.21.15.105:0/2416308136 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f143809ebf0 con 0x7f1438161850 2024-03-08T08:29:52.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.973+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1438075120 con 0x7f1438161850 2024-03-08T08:29:52.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.974+0000 7f1434ff9640 1 -- 172.21.15.105:0/2416308136 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1430058d70 con 0x7f1438161850 2024-03-08T08:29:52.975 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f142406b4d0 msgr2=0x7f142406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f142406b4d0 0x7f142406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1428036000 tx=0x7f1428008030 comp rx=0 tx=0).stop 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161850 msgr2=0x7f14380a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161850 0x7f14380a12c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f143000b9e0 tx=0x7f143000bea0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 shutdown_connections 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f142406b4d0 0x7f142406d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1438161850 0x7f14380a12c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1438161470 0x7f14381f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 --2- 172.21.15.105:0/2416308136 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f143806bcb0 0x7f143815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 >> 172.21.15.105:0/2416308136 conn(0x7f14380d3ce0 msgr2=0x7f14380d40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 shutdown_connections 2024-03-08T08:29:52.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:52.976+0000 7f143e948640 1 -- 172.21.15.105:0/2416308136 wait complete. 2024-03-08T08:29:52.987 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T08:29:53.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.089+0000 7f69ed664640 1 Processor -- start 2024-03-08T08:29:53.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.089+0000 7f69ed664640 1 -- start start 2024-03-08T08:29:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69ed664640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69e8161470 0x7f69e806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69ed664640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69e8161850 0x7f69e80d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69ed664640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e80d48a0 con 0x7f69e806d370 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69ed664640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e80d2ba0 con 0x7f69e8161470 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69ed664640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80d4b80 con 0x7f69e8161850 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69e77fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f69e806d370 0x7f69e806b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56992/0 (socket says 172.21.15.105:56992) 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69e77fe640 1 -- 172.21.15.105:0/3589491218 learned_addr learned my addr 172.21.15.105:0/3589491218 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69e67fc640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69e8161850 0x7f69e80d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69e6ffd640 1 --2- 172.21.15.105:0/3589491218 >> v2:172.21.15.144:3300/0 conn(0x7f69e8161470 0x7f69e806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164084889 0 0) 0x7f69e80d48a0 con 0x7f69e806d370 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.090+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d0003520 con 0x7f69e806d370 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4278376322 0 0) 0x7f69d0003520 con 0x7f69e806d370 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69e8161850 msgr2=0x7f69e80d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69e8161850 0x7f69e80d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 >> v2:172.21.15.144:3300/0 conn(0x7f69e8161470 msgr2=0x7f69e806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 --2- 172.21.15.105:0/3589491218 >> v2:172.21.15.144:3300/0 conn(0x7f69e8161470 0x7f69e806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e806c170 con 0x7f69e806d370 2024-03-08T08:29:53.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f69e00019f0 con 0x7f69e806d370 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 >> v1:172.21.15.105:6789/0 conn(0x7f69e806d370 legacy=0x7f69e806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69d00039b0 0x7f69d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69d00051c0 0x7f69d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80d2ba0 con 0x7f69d0004800 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80d4b80 con 0x7f69d00039b0 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d000bf30 con 0x7f69d00051c0 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e6ffd640 1 --2- 172.21.15.105:0/3589491218 >> v2:172.21.15.144:3300/0 conn(0x7f69e8161470 0x7f69e806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e67fc640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69e8161850 0x7f69e80d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e67fc640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e77fe640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69d00051c0 0x7f69d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e6ffd640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69d00039b0 0x7f69d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e67fc640 1 -- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69d00051c0 msgr2=0x7f69d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e67fc640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69d00051c0 0x7f69d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e67fc640 1 -- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69d00039b0 msgr2=0x7f69d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e67fc640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69d00039b0 0x7f69d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.091+0000 7f69e67fc640 1 -- 172.21.15.105:0/3589491218 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0003520 con 0x7f69d0004800 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.092+0000 7f69e67fc640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009b00 tx=0x7f69d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9048c456f4235 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.092+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0097d0 con 0x7f69d0004800 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.092+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc004030 con 0x7f69d0004800 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.092+0000 7f69e5ffb640 1 -- 172.21.15.105:0/3589491218 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc004400 con 0x7f69d0004800 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.093+0000 7f69ed664640 1 -- 172.21.15.105:0/3589491218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d0004800 msgr2=0x7f69d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.093+0000 7f69ed664640 1 --2- 172.21.15.105:0/3589491218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009b00 tx=0x7f69d0005830 comp rx=0 tx=0).stop 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.093+0000 7f69e77fe640 1 -- 172.21.15.105:0/3589491218 reap_dead start 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.093+0000 7f69ed664640 1 -- 172.21.15.105:0/3589491218 shutdown_connections 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.093+0000 7f69ed664640 1 -- 172.21.15.105:0/3589491218 >> 172.21.15.105:0/3589491218 conn(0x7f69e80d3ce0 msgr2=0x7f69e80d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.093+0000 7f69ed664640 1 -- 172.21.15.105:0/3589491218 shutdown_connections 2024-03-08T08:29:53.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.093+0000 7f69ed664640 1 -- 172.21.15.105:0/3589491218 wait complete. 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.094+0000 7f69ed664640 1 Processor -- start 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.094+0000 7f69ed664640 1 -- start start 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69ed664640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69d0004800 0x7f69e80cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69ed664640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e815c620 0x7f69e80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69ed664640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e80b90b0 0x7f69e80b9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69ed664640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e806cf40 con 0x7f69e80b90b0 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69ed664640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e806bcb0 con 0x7f69e815c620 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69ed664640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80b1a70 con 0x7f69d0004800 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e80b90b0 0x7f69e80b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e80b90b0 0x7f69e80b9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56540/0 (socket says 172.21.15.105:56540) 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 -- 172.21.15.105:0/2107698752 learned_addr learned my addr 172.21.15.105:0/2107698752 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 -- 172.21.15.105:0/2107698752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69d0004800 msgr2=0x7f69e80cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69d0004800 0x7f69e80cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 -- 172.21.15.105:0/2107698752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e815c620 msgr2=0x7f69e80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e67fc640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e815c620 0x7f69e80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e815c620 0x7f69e80cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 -- 172.21.15.105:0/2107698752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e80cbf50 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69e77fe640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e80b90b0 0x7f69e80b9530 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f69e0007600 tx=0x7f69e0007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69cf7fe640 1 -- 172.21.15.105:0/2107698752 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000ab90 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69cf7fe640 1 -- 172.21.15.105:0/2107698752 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0018070 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69cf7fe640 1 -- 172.21.15.105:0/2107698752 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00133f0 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e80cc230 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.095+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e80cdf00 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.096+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e8075120 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.097+0000 7f69cf7fe640 1 -- 172.21.15.105:0/2107698752 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69e0013590 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.097+0000 7f69cf7fe640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f69d406b4d0 0x7f69d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.097+0000 7f69e6ffd640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f69d406b4d0 0x7f69d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.097+0000 7f69cf7fe640 1 -- 172.21.15.105:0/2107698752 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69e008a8a0 con 0x7f69e80b90b0 2024-03-08T08:29:53.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.098+0000 7f69e6ffd640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f69d406b4d0 0x7f69d406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009c40 tx=0x7f69d8009760 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.102+0000 7f69cf7fe640 1 -- 172.21.15.105:0/2107698752 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69e0059720 con 0x7f69e80b90b0 2024-03-08T08:29:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.272+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69e80d1b30 con 0x7f69e80b90b0 2024-03-08T08:29:53.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.273+0000 7f69cf7fe640 1 -- 172.21.15.105:0/2107698752 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f69e0059020 con 0x7f69e80b90b0 2024-03-08T08:29:53.274 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-08T08:29:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f69d406b4d0 msgr2=0x7f69d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f69d406b4d0 0x7f69d406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009c40 tx=0x7f69d8009760 comp rx=0 tx=0).stop 2024-03-08T08:29:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e80b90b0 msgr2=0x7f69e80b9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e80b90b0 0x7f69e80b9530 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f69e0007600 tx=0x7f69e0007ac0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 shutdown_connections 2024-03-08T08:29:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f69d406b4d0 0x7f69d406d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69e80b90b0 0x7f69e80b9530 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69e815c620 0x7f69e80cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 --2- 172.21.15.105:0/2107698752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69d0004800 0x7f69e80cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 >> 172.21.15.105:0/2107698752 conn(0x7f69e80d3ce0 msgr2=0x7f69e815d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 shutdown_connections 2024-03-08T08:29:53.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.275+0000 7f69ed664640 1 -- 172.21.15.105:0/2107698752 wait complete. 2024-03-08T08:29:53.287 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:29:53.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.385+0000 7fdfcf846640 1 Processor -- start 2024-03-08T08:29:53.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.386+0000 7fdfcf846640 1 -- start start 2024-03-08T08:29:53.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.386+0000 7fdfcf846640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdfc815dfa0 0x7fdfc806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfcf846640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdfc806b040 0x7fdfc80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfcf846640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc80adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfcf846640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc815e380 con 0x7fdfc806b040 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfcf846640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc80b5f60 con 0x7fdfc815dfa0 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfcf846640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80b6240 con 0x7fdfc80ad740 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfccde1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdfc806b040 0x7fdfc80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfccde1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdfc806b040 0x7fdfc80ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56544/0 (socket says 172.21.15.105:56544) 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfccde1640 1 -- 172.21.15.105:0/3626179747 learned_addr learned my addr 172.21.15.105:0/3626179747 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfcd5e2640 1 --2- 172.21.15.105:0/3626179747 >> v2:172.21.15.144:3300/0 conn(0x7fdfc815dfa0 0x7fdfc806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfcdde3640 1 --2- 172.21.15.105:0/3626179747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc80adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfccde1640 1 -- 172.21.15.105:0/3626179747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 msgr2=0x7fdfc80adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfccde1640 1 --2- 172.21.15.105:0/3626179747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc80adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfccde1640 1 -- 172.21.15.105:0/3626179747 >> v2:172.21.15.144:3300/0 conn(0x7fdfc815dfa0 msgr2=0x7fdfc806c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfccde1640 1 --2- 172.21.15.105:0/3626179747 >> v2:172.21.15.144:3300/0 conn(0x7fdfc815dfa0 0x7fdfc806c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.387+0000 7fdfccde1640 1 -- 172.21.15.105:0/3626179747 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc806b720 con 0x7fdfc806b040 2024-03-08T08:29:53.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfccde1640 1 --2- 172.21.15.105:0/3626179747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc80ad200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8073360 tx=0x7fdfc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b68d93f139ed2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfbffff640 1 -- 172.21.15.105:0/3626179747 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4035020 con 0x7fdfc806b040 2024-03-08T08:29:53.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfbffff640 1 -- 172.21.15.105:0/3626179747 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc4031870 con 0x7fdfc806b040 2024-03-08T08:29:53.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfbffff640 1 -- 172.21.15.105:0/3626179747 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4031c40 con 0x7fdfc806b040 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfcf846640 1 -- 172.21.15.105:0/3626179747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 msgr2=0x7fdfc80ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfcf846640 1 --2- 172.21.15.105:0/3626179747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc80ad200 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8073360 tx=0x7fdfc402b910 comp rx=0 tx=0).stop 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfcf846640 1 -- 172.21.15.105:0/3626179747 shutdown_connections 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfcf846640 1 --2- 172.21.15.105:0/3626179747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc80adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfcf846640 1 --2- 172.21.15.105:0/3626179747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc80ad200 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfcf846640 1 --2- 172.21.15.105:0/3626179747 >> v2:172.21.15.144:3300/0 conn(0x7fdfc815dfa0 0x7fdfc806c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.388+0000 7fdfcf846640 1 -- 172.21.15.105:0/3626179747 >> 172.21.15.105:0/3626179747 conn(0x7fdfc80b50f0 msgr2=0x7fdfc80b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.389+0000 7fdfcf846640 1 -- 172.21.15.105:0/3626179747 shutdown_connections 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.389+0000 7fdfcf846640 1 -- 172.21.15.105:0/3626179747 wait complete. 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcf846640 1 Processor -- start 2024-03-08T08:29:53.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcf846640 1 -- start start 2024-03-08T08:29:53.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcf846640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc81e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcf846640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc81e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcf846640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfc815dfa0 0x7fdfc81e86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcf846640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806d000 con 0x7fdfc806b040 2024-03-08T08:29:53.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcf846640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806bce0 con 0x7fdfc815dfa0 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcf846640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806d180 con 0x7fdfc80ad740 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.390+0000 7fdfcd5e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc81e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcd5e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc81e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56552/0 (socket says 172.21.15.105:56552) 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcd5e2640 1 -- 172.21.15.105:0/1722761011 learned_addr learned my addr 172.21.15.105:0/1722761011 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcdde3640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfc815dfa0 0x7fdfc81e86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfccde1640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc81e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcd5e2640 1 -- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 msgr2=0x7fdfc81e81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcd5e2640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc81e81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcd5e2640 1 -- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfc815dfa0 msgr2=0x7fdfc81e86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcd5e2640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfc815dfa0 0x7fdfc81e86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcd5e2640 1 -- 172.21.15.105:0/1722761011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80b1230 con 0x7fdfc806b040 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfccde1640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc81e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcd5e2640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc81e78a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800d5f0 tx=0x7fdfb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfbdffb640 1 -- 172.21.15.105:0/1722761011 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80125f0 con 0x7fdfc806b040 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfbdffb640 1 -- 172.21.15.105:0/1722761011 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb8019070 con 0x7fdfc806b040 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc80b2a10 con 0x7fdfc806b040 2024-03-08T08:29:53.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfbdffb640 1 -- 172.21.15.105:0/1722761011 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80129d0 con 0x7fdfc806b040 2024-03-08T08:29:53.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.391+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc8140fb0 con 0x7fdfc806b040 2024-03-08T08:29:53.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.392+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc80686c0 con 0x7fdfc806b040 2024-03-08T08:29:53.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.393+0000 7fdfbdffb640 1 -- 172.21.15.105:0/1722761011 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfb8012b70 con 0x7fdfc806b040 2024-03-08T08:29:53.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.393+0000 7fdfbdffb640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdfac06b500 0x7fdfac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.393+0000 7fdfbdffb640 1 -- 172.21.15.105:0/1722761011 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfb808a5a0 con 0x7fdfc806b040 2024-03-08T08:29:53.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.393+0000 7fdfccde1640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdfac06b500 0x7fdfac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.393+0000 7fdfccde1640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdfac06b500 0x7fdfac06d9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4036000 tx=0x7fdfc4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.397+0000 7fdfbdffb640 1 -- 172.21.15.105:0/1722761011 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfc80686c0 con 0x7fdfc806b040 2024-03-08T08:29:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.604+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfc8075120 con 0x7fdfc806b040 2024-03-08T08:29:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.604+0000 7fdfbdffb640 1 -- 172.21.15.105:0/1722761011 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fdfb8059420 con 0x7fdfc806b040 2024-03-08T08:29:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:53.606 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2759286099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2759286099},{"type":"v1","addr":"172.21.15.144:6834","nonce":2759286099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2788852991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2788852991},{"type":"v1","addr":"172.21.15.144:6835","nonce":2788852991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2226664928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2226664928},{"type":"v1","addr":"172.21.15.105:6835","nonce":2226664928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:28:31.203118+0000","modified":"2024-03-08T08:28:33.213155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2431299564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2431299564},{"type":"v1","addr":"172.21.15.105:6837","nonce":2431299564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T08:29:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 13 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdfac06b500 msgr2=0x7fdfac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdfac06b500 0x7fdfac06d9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4036000 tx=0x7fdfc4008030 comp rx=0 tx=0).stop 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 msgr2=0x7fdfc81e78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc81e78a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800d5f0 tx=0x7fdfb800dab0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 shutdown_connections 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdfac06b500 0x7fdfac06d9c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfc815dfa0 0x7fdfc81e86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfc80ad740 0x7fdfc81e81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 --2- 172.21.15.105:0/1722761011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfc806b040 0x7fdfc81e78a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.606+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 >> 172.21.15.105:0/1722761011 conn(0x7fdfc80b50f0 msgr2=0x7fdfc815c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.607+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 shutdown_connections 2024-03-08T08:29:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.607+0000 7fdfcf846640 1 -- 172.21.15.105:0/1722761011 wait complete. 2024-03-08T08:29:53.618 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:29:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.717+0000 7f37cb748640 1 Processor -- start 2024-03-08T08:29:53.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.717+0000 7f37cb748640 1 -- start start 2024-03-08T08:29:53.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37cb748640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c4161520 0x7f37c40b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37cb748640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f37c40ce790 0x7f37c40b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37cb748640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c40b29e0 con 0x7f37c40ce330 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37cb748640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c40d1930 con 0x7f37c40ce790 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37cb748640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c406c770 con 0x7f37c4161520 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37c9ce5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f37c40ce790 0x7f37c40b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37c94e4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c4161520 0x7f37c40b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37c8ce3640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f37c40ce330 0x7f37c40b14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:58770/0 (socket says 172.21.15.105:58770) 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37c9ce5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f37c40ce790 0x7f37c40b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49116/0 (socket says 172.21.15.105:49116) 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37c8ce3640 1 -- 172.21.15.105:0/3961328899 learned_addr learned my addr 172.21.15.105:0/3961328899 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37c9ce5640 1 -- 172.21.15.105:0/3961328899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c4161520 msgr2=0x7f37c40b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.718+0000 7f37c9ce5640 1 --2- 172.21.15.105:0/3961328899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c4161520 0x7f37c40b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.719+0000 7f37c9ce5640 1 -- 172.21.15.105:0/3961328899 >> v1:172.21.15.144:6789/0 conn(0x7f37c40ce330 legacy=0x7f37c40b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.719+0000 7f37bbfff640 1 -- 172.21.15.105:0/3961328899 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21914473 0 0) 0x7f37c40b29e0 con 0x7f37c40ce330 2024-03-08T08:29:53.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.719+0000 7f37c9ce5640 1 -- 172.21.15.105:0/3961328899 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40d1650 con 0x7f37c40ce790 2024-03-08T08:29:53.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.719+0000 7f37c9ce5640 1 --2- 172.21.15.105:0/3961328899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c40ce790 0x7f37c40b1c00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009d90 tx=0x7f37bc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c93554a5f3bec8dd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.719+0000 7f37bbfff640 1 -- 172.21.15.105:0/3961328899 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc039070 con 0x7f37c40ce790 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.719+0000 7f37bbfff640 1 -- 172.21.15.105:0/3961328899 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc02cc50 con 0x7f37c40ce790 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37bbfff640 1 -- 172.21.15.105:0/3961328899 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc034440 con 0x7f37c40ce790 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37cb748640 1 -- 172.21.15.105:0/3961328899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c40ce790 msgr2=0x7f37c40b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37cb748640 1 --2- 172.21.15.105:0/3961328899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c40ce790 0x7f37c40b1c00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009d90 tx=0x7f37bc02bea0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37cb748640 1 -- 172.21.15.105:0/3961328899 shutdown_connections 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37cb748640 1 --2- 172.21.15.105:0/3961328899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c40ce790 0x7f37c40b1c00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37cb748640 1 --2- 172.21.15.105:0/3961328899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c4161520 0x7f37c40b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37cb748640 1 -- 172.21.15.105:0/3961328899 >> 172.21.15.105:0/3961328899 conn(0x7f37c40b36a0 msgr2=0x7f37c40b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37cb748640 1 -- 172.21.15.105:0/3961328899 shutdown_connections 2024-03-08T08:29:53.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.720+0000 7f37cb748640 1 -- 172.21.15.105:0/3961328899 wait complete. 2024-03-08T08:29:53.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.721+0000 7f37cb748640 1 Processor -- start 2024-03-08T08:29:53.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.721+0000 7f37cb748640 1 -- start start 2024-03-08T08:29:53.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37cb748640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c40ce330 0x7f37c40cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37cb748640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c40ce790 0x7f37c40b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37cb748640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c4161520 0x7f37c40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37cb748640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4161900 con 0x7f37c40ce790 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37cb748640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c406ca40 con 0x7f37c4161520 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37cb748640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40b32d0 con 0x7f37c40ce330 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c8ce3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c40ce790 0x7f37c40b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c8ce3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c40ce790 0x7f37c40b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44486/0 (socket says 172.21.15.105:44486) 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c8ce3640 1 -- 172.21.15.105:0/2060953132 learned_addr learned my addr 172.21.15.105:0/2060953132 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c9ce5640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c4161520 0x7f37c40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c94e4640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c40ce330 0x7f37c40cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c8ce3640 1 -- 172.21.15.105:0/2060953132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c40ce330 msgr2=0x7f37c40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c8ce3640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c40ce330 0x7f37c40cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c8ce3640 1 -- 172.21.15.105:0/2060953132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c4161520 msgr2=0x7f37c40b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c8ce3640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c4161520 0x7f37c40b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.722+0000 7f37c8ce3640 1 -- 172.21.15.105:0/2060953132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40b4d90 con 0x7f37c40ce790 2024-03-08T08:29:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.723+0000 7f37c8ce3640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c40ce790 0x7f37c40b43d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f37b400c4d0 tx=0x7f37b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.723+0000 7f37b9ffb640 1 -- 172.21.15.105:0/2060953132 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4018070 con 0x7f37c40ce790 2024-03-08T08:29:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.723+0000 7f37b9ffb640 1 -- 172.21.15.105:0/2060953132 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b4007500 con 0x7f37c40ce790 2024-03-08T08:29:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.723+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c40b5070 con 0x7f37c40ce790 2024-03-08T08:29:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.723+0000 7f37b9ffb640 1 -- 172.21.15.105:0/2060953132 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4004a20 con 0x7f37c40ce790 2024-03-08T08:29:53.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.723+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c40b5910 con 0x7f37c40ce790 2024-03-08T08:29:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.724+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c40b5240 con 0x7f37c40ce790 2024-03-08T08:29:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.724+0000 7f37b9ffb640 1 -- 172.21.15.105:0/2060953132 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37b401b070 con 0x7f37c40ce790 2024-03-08T08:29:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.725+0000 7f37b9ffb640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37a806b4d0 0x7f37a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.725+0000 7f37b9ffb640 1 -- 172.21.15.105:0/2060953132 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37b4089490 con 0x7f37c40ce790 2024-03-08T08:29:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.725+0000 7f37c94e4640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37a806b4d0 0x7f37a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:53.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.725+0000 7f37c94e4640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37a806b4d0 0x7f37a806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009aa0 tx=0x7f37c0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:53.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.729+0000 7f37b9ffb640 1 -- 172.21.15.105:0/2060953132 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37b4057cc0 con 0x7f37c40ce790 2024-03-08T08:29:53.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.898+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37c40c8b60 con 0x7f37c40ce790 2024-03-08T08:29:53.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.899+0000 7f37b9ffb640 1 -- 172.21.15.105:0/2060953132 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16681 (secure 0 0 0) 0x7f37c40c8b60 con 0x7f37c40ce790 2024-03-08T08:29:53.901 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:53.901 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":33,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:32.876978+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:28:29.680862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T08:28:30.179255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37a806b4d0 msgr2=0x7f37a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37a806b4d0 0x7f37a806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009aa0 tx=0x7f37c0009190 comp rx=0 tx=0).stop 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c40ce790 msgr2=0x7f37c40b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c40ce790 0x7f37c40b43d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f37b400c4d0 tx=0x7f37b400c990 comp rx=0 tx=0).stop 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 shutdown_connections 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f37a806b4d0 0x7f37a806d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c4161520 0x7f37c40b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37c40ce790 0x7f37c40b43d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 --2- 172.21.15.105:0/2060953132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c40ce330 0x7f37c40cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.901+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 >> 172.21.15.105:0/2060953132 conn(0x7f37c40b36a0 msgr2=0x7f37c40b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.902+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 shutdown_connections 2024-03-08T08:29:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:53.902+0000 7f37cb748640 1 -- 172.21.15.105:0/2060953132 wait complete. 2024-03-08T08:29:53.913 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T08:29:53.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:29:54.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.013+0000 7fe9ccd6c640 1 Processor -- start 2024-03-08T08:29:54.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.013+0000 7fe9ccd6c640 1 -- start start 2024-03-08T08:29:54.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9ccd6c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe9c806b040 0x7fe9c806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9ccd6c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe9c80ad910 0x7fe9c80d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9ccd6c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c815e380 con 0x7fe9c815dfa0 2024-03-08T08:29:54.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9ccd6c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c80b5f60 con 0x7fe9c806b040 2024-03-08T08:29:54.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9ccd6c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c80b6240 con 0x7fe9c80ad910 2024-03-08T08:29:54.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9bffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe9c806b040 0x7fe9c806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9c659c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe9c815dfa0 0x7fe9c80ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51790/0 (socket says 172.21.15.105:51790) 2024-03-08T08:29:54.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9c659c640 1 -- 172.21.15.105:0/1303770101 learned_addr learned my addr 172.21.15.105:0/1303770101 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:54.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9bffff640 1 -- 172.21.15.105:0/1303770101 >> v2:172.21.15.144:3300/0 conn(0x7fe9c80ad910 msgr2=0x7fe9c80d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:54.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9bffff640 1 --2- 172.21.15.105:0/1303770101 >> v2:172.21.15.144:3300/0 conn(0x7fe9c80ad910 0x7fe9c80d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9bffff640 1 -- 172.21.15.105:0/1303770101 >> v1:172.21.15.105:6789/0 conn(0x7fe9c815dfa0 legacy=0x7fe9c80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.014+0000 7fe9bffff640 1 -- 172.21.15.105:0/1303770101 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c806b720 con 0x7fe9c806b040 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.015+0000 7fe9bffff640 1 --2- 172.21.15.105:0/1303770101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c806c200 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8070e80 tx=0x7fe9b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc94fba36009c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.015+0000 7fe9c5d9b640 1 -- 172.21.15.105:0/1303770101 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0035020 con 0x7fe9c806b040 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.015+0000 7fe9c5d9b640 1 -- 172.21.15.105:0/1303770101 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0031830 con 0x7fe9c806b040 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.015+0000 7fe9c5d9b640 1 -- 172.21.15.105:0/1303770101 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0031c00 con 0x7fe9c806b040 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.015+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1303770101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 msgr2=0x7fe9c806c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.015+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1303770101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c806c200 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8070e80 tx=0x7fe9b002b8d0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.015+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1303770101 shutdown_connections 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.016+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1303770101 >> v2:172.21.15.144:3300/0 conn(0x7fe9c80ad910 0x7fe9c80d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.016+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1303770101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c806c200 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.016+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1303770101 >> 172.21.15.105:0/1303770101 conn(0x7fe9c80b50f0 msgr2=0x7fe9c80b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:54.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.016+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1303770101 shutdown_connections 2024-03-08T08:29:54.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.016+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1303770101 wait complete. 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.017+0000 7fe9ccd6c640 1 Processor -- start 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.017+0000 7fe9ccd6c640 1 -- start start 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.017+0000 7fe9ccd6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c8158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9ccd6c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9c80ad910 0x7fe9c8160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9ccd6c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c815dfa0 0x7fe9c809a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9ccd6c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c806d000 con 0x7fe9c806b040 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9ccd6c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c806bce0 con 0x7fe9c815dfa0 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9ccd6c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c806d180 con 0x7fe9c80ad910 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c8158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44504/0 (socket says 172.21.15.105:44504) 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 -- 172.21.15.105:0/1552942854 learned_addr learned my addr 172.21.15.105:0/1552942854 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 -- 172.21.15.105:0/1552942854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9c80ad910 msgr2=0x7fe9c8160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9c80ad910 0x7fe9c8160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 -- 172.21.15.105:0/1552942854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c815dfa0 msgr2=0x7fe9c809a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c815dfa0 0x7fe9c809a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 -- 172.21.15.105:0/1552942854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c815faa0 con 0x7fe9c806b040 2024-03-08T08:29:54.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9c659c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c8158150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c3c0 tx=0x7fe9b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9beffd640 1 -- 172.21.15.105:0/1552942854 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8017070 con 0x7fe9c806b040 2024-03-08T08:29:54.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9beffd640 1 -- 172.21.15.105:0/1552942854 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8005ce0 con 0x7fe9c806b040 2024-03-08T08:29:54.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c815fd80 con 0x7fe9c806b040 2024-03-08T08:29:54.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9beffd640 1 -- 172.21.15.105:0/1552942854 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80072f0 con 0x7fe9c806b040 2024-03-08T08:29:54.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.018+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c8160340 con 0x7fe9c806b040 2024-03-08T08:29:54.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.019+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c8066c00 con 0x7fe9c806b040 2024-03-08T08:29:54.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.020+0000 7fe9beffd640 1 -- 172.21.15.105:0/1552942854 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9b801a070 con 0x7fe9c806b040 2024-03-08T08:29:54.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.020+0000 7fe9beffd640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe9a806b4d0 0x7fe9a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.020+0000 7fe9bffff640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe9a806b4d0 0x7fe9a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.020+0000 7fe9beffd640 1 -- 172.21.15.105:0/1552942854 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9b80895a0 con 0x7fe9c806b040 2024-03-08T08:29:54.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.021+0000 7fe9bffff640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe9a806b4d0 0x7fe9a806d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0036000 tx=0x7fe9b0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.024+0000 7fe9beffd640 1 -- 172.21.15.105:0/1552942854 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9c8066c00 con 0x7fe9c806b040 2024-03-08T08:29:54.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.230+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9c8075120 con 0x7fe9c806b040 2024-03-08T08:29:54.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.230+0000 7fe9beffd640 1 -- 172.21.15.105:0/1552942854 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fe9b8058420 con 0x7fe9c806b040 2024-03-08T08:29:54.231 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:54.231 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2759286099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2759286099},{"type":"v1","addr":"172.21.15.144:6834","nonce":2759286099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2788852991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2788852991},{"type":"v1","addr":"172.21.15.144:6835","nonce":2788852991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2226664928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2226664928},{"type":"v1","addr":"172.21.15.105:6835","nonce":2226664928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:28:31.203118+0000","modified":"2024-03-08T08:28:33.213155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2431299564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2431299564},{"type":"v1","addr":"172.21.15.105:6837","nonce":2431299564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T08:29:54.232 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 13 2024-03-08T08:29:54.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe9a806b4d0 msgr2=0x7fe9a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe9a806b4d0 0x7fe9a806d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0036000 tx=0x7fe9b0030000 comp rx=0 tx=0).stop 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 msgr2=0x7fe9c8158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c8158150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c3c0 tx=0x7fe9b800c880 comp rx=0 tx=0).stop 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 shutdown_connections 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe9a806b4d0 0x7fe9a806d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9c815dfa0 0x7fe9c809a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9c80ad910 0x7fe9c8160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 --2- 172.21.15.105:0/1552942854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9c806b040 0x7fe9c8158150 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 >> 172.21.15.105:0/1552942854 conn(0x7fe9c80b50f0 msgr2=0x7fe9c815bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 shutdown_connections 2024-03-08T08:29:54.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.232+0000 7fe9ccd6c640 1 -- 172.21.15.105:0/1552942854 wait complete. 2024-03-08T08:29:54.244 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:29:54.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.342+0000 7fb55f846640 1 Processor -- start 2024-03-08T08:29:54.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.342+0000 7fb55f846640 1 -- start start 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55f846640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55815aad0 0x7fb55815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55f846640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb55815aeb0 0x7fb55806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55f846640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb55806ec20 0x7fb55806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55f846640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55815ebb0 con 0x7fb55806ec20 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55f846640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55815ee90 con 0x7fb55815aeb0 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55f846640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55815f170 con 0x7fb55815aad0 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55dde3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb55806ec20 0x7fb55806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55dde3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb55806ec20 0x7fb55806d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44510/0 (socket says 172.21.15.105:44510) 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.343+0000 7fb55dde3640 1 -- 172.21.15.105:0/2529813088 learned_addr learned my addr 172.21.15.105:0/2529813088 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55dde3640 1 -- 172.21.15.105:0/2529813088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55815aad0 msgr2=0x7fb55815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:54.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55d5e2640 1 --2- 172.21.15.105:0/2529813088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55815aad0 0x7fb55815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55cde1640 1 --2- 172.21.15.105:0/2529813088 >> v2:172.21.15.144:3300/0 conn(0x7fb55815aeb0 0x7fb55806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55dde3640 1 --2- 172.21.15.105:0/2529813088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55815aad0 0x7fb55815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55dde3640 1 -- 172.21.15.105:0/2529813088 >> v2:172.21.15.144:3300/0 conn(0x7fb55815aeb0 msgr2=0x7fb55806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55dde3640 1 --2- 172.21.15.105:0/2529813088 >> v2:172.21.15.144:3300/0 conn(0x7fb55815aeb0 0x7fb55806e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55dde3640 1 -- 172.21.15.105:0/2529813088 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55815f450 con 0x7fb55806ec20 2024-03-08T08:29:54.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55cde1640 1 --2- 172.21.15.105:0/2529813088 >> v2:172.21.15.144:3300/0 conn(0x7fb55815aeb0 0x7fb55806e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:29:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb55dde3640 1 --2- 172.21.15.105:0/2529813088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55806ec20 0x7fb55806d4c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb558073490 tx=0x7fb55002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3655ca139c302d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.344+0000 7fb54ffff640 1 -- 172.21.15.105:0/2529813088 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55002ca70 con 0x7fb55806ec20 2024-03-08T08:29:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb54ffff640 1 -- 172.21.15.105:0/2529813088 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55002cc10 con 0x7fb55806ec20 2024-03-08T08:29:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb54ffff640 1 -- 172.21.15.105:0/2529813088 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5500343f0 con 0x7fb55806ec20 2024-03-08T08:29:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 -- 172.21.15.105:0/2529813088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55806ec20 msgr2=0x7fb55806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 --2- 172.21.15.105:0/2529813088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55806ec20 0x7fb55806d4c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb558073490 tx=0x7fb55002bed0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 -- 172.21.15.105:0/2529813088 shutdown_connections 2024-03-08T08:29:54.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 --2- 172.21.15.105:0/2529813088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55806ec20 0x7fb55806d4c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 --2- 172.21.15.105:0/2529813088 >> v2:172.21.15.144:3300/0 conn(0x7fb55815aeb0 0x7fb55806e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 --2- 172.21.15.105:0/2529813088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55815aad0 0x7fb55815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 -- 172.21.15.105:0/2529813088 >> 172.21.15.105:0/2529813088 conn(0x7fb55806bd90 msgr2=0x7fb55806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 -- 172.21.15.105:0/2529813088 shutdown_connections 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.345+0000 7fb55f846640 1 -- 172.21.15.105:0/2529813088 wait complete. 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.346+0000 7fb55f846640 1 Processor -- start 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.346+0000 7fb55f846640 1 -- start start 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55f846640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55806ec20 0x7fb55806aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55f846640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55815aad0 0x7fb55814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55f846640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb55815aeb0 0x7fb55814fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55f846640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5580728a0 con 0x7fb55815aad0 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55f846640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55815b330 con 0x7fb55815aeb0 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55f846640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55806da00 con 0x7fb55806ec20 2024-03-08T08:29:54.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55cde1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55815aad0 0x7fb55814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55cde1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55815aad0 0x7fb55814f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44524/0 (socket says 172.21.15.105:44524) 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55cde1640 1 -- 172.21.15.105:0/4022188426 learned_addr learned my addr 172.21.15.105:0/4022188426 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55cde1640 1 -- 172.21.15.105:0/4022188426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55806ec20 msgr2=0x7fb55806aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55cde1640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55806ec20 0x7fb55806aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55cde1640 1 -- 172.21.15.105:0/4022188426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb55815aeb0 msgr2=0x7fb55814fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.347+0000 7fb55dde3640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb55815aeb0 0x7fb55814fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.348+0000 7fb55cde1640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb55815aeb0 0x7fb55814fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.348+0000 7fb55cde1640 1 -- 172.21.15.105:0/4022188426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5581f4af0 con 0x7fb55815aad0 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.348+0000 7fb55cde1640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55815aad0 0x7fb55814f700 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c7f0 tx=0x7fb54800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.348+0000 7fb54dffb640 1 -- 172.21.15.105:0/4022188426 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548015070 con 0x7fb55815aad0 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.348+0000 7fb54dffb640 1 -- 172.21.15.105:0/4022188426 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb548007820 con 0x7fb55815aad0 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.348+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5581f4dd0 con 0x7fb55815aad0 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.348+0000 7fb54dffb640 1 -- 172.21.15.105:0/4022188426 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548007bd0 con 0x7fb55815aad0 2024-03-08T08:29:54.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.348+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5581f5390 con 0x7fb55815aad0 2024-03-08T08:29:54.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.349+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb558075120 con 0x7fb55815aad0 2024-03-08T08:29:54.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.349+0000 7fb54dffb640 1 -- 172.21.15.105:0/4022188426 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb548018070 con 0x7fb55815aad0 2024-03-08T08:29:54.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.350+0000 7fb54dffb640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb53c06b500 0x7fb53c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.350+0000 7fb55d5e2640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb53c06b500 0x7fb53c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.350+0000 7fb54dffb640 1 -- 172.21.15.105:0/4022188426 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb54808a280 con 0x7fb55815aad0 2024-03-08T08:29:54.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.350+0000 7fb55d5e2640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb53c06b500 0x7fb53c06d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb554006b10 tx=0x7fb554008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.354+0000 7fb54dffb640 1 -- 172.21.15.105:0/4022188426 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb548059190 con 0x7fb55815aad0 2024-03-08T08:29:54.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.561+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5580cd210 con 0x7fb55815aad0 2024-03-08T08:29:54.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.561+0000 7fb54dffb640 1 -- 172.21.15.105:0/4022188426 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fb548058a90 con 0x7fb55815aad0 2024-03-08T08:29:54.562 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:54.562 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2759286099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2759286099},{"type":"v1","addr":"172.21.15.144:6834","nonce":2759286099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2788852991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2788852991},{"type":"v1","addr":"172.21.15.144:6835","nonce":2788852991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2226664928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2226664928},{"type":"v1","addr":"172.21.15.105:6835","nonce":2226664928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-08T08:28:31.203118+0000","modified":"2024-03-08T08:28:33.213155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6837/2431299564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2431299564},{"type":"v1","addr":"172.21.15.105:6837","nonce":2431299564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-08T08:29:54.563 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 13 2024-03-08T08:29:54.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb53c06b500 msgr2=0x7fb53c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb53c06b500 0x7fb53c06d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb554006b10 tx=0x7fb554008040 comp rx=0 tx=0).stop 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55815aad0 msgr2=0x7fb55814f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55815aad0 0x7fb55814f700 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c7f0 tx=0x7fb54800ccb0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 shutdown_connections 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb53c06b500 0x7fb53c06d9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb55815aeb0 0x7fb55814fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb55815aad0 0x7fb55814f700 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 --2- 172.21.15.105:0/4022188426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb55806ec20 0x7fb55806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 >> 172.21.15.105:0/4022188426 conn(0x7fb55806bd90 msgr2=0x7fb5580b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 shutdown_connections 2024-03-08T08:29:54.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.563+0000 7fb55f846640 1 -- 172.21.15.105:0/4022188426 wait complete. 2024-03-08T08:29:54.575 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:29:54.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.673+0000 7ff61858c640 1 Processor -- start 2024-03-08T08:29:54.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.673+0000 7ff61858c640 1 -- start start 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.674+0000 7ff61858c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 0x7ff610071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.674+0000 7ff61858c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6100b4d00 0x7ff61015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.674+0000 7ff61858c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61015b940 con 0x7ff61015b4a0 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.674+0000 7ff61858c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61015bc20 con 0x7ff6100b4d00 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.674+0000 7ff61858c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff610070840 con 0x7ff610072440 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.675+0000 7ff615b27640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6100b4d00 0x7ff61015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.675+0000 7ff615b27640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff6100b4d00 0x7ff61015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44526/0 (socket says 172.21.15.105:44526) 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.675+0000 7ff615b27640 1 -- 172.21.15.105:0/2251796105 learned_addr learned my addr 172.21.15.105:0/2251796105 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:54.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.675+0000 7ff615b27640 1 -- 172.21.15.105:0/2251796105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 msgr2=0x7ff610071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.675+0000 7ff615b27640 1 --2- 172.21.15.105:0/2251796105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 0x7ff610071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.675+0000 7ff615b27640 1 -- 172.21.15.105:0/2251796105 >> v1:172.21.15.144:6789/0 conn(0x7ff61015b4a0 legacy=0x7ff61006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:54.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.675+0000 7ff615b27640 1 -- 172.21.15.105:0/2251796105 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610070b20 con 0x7ff6100b4d00 2024-03-08T08:29:54.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff615b27640 1 --2- 172.21.15.105:0/2251796105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6100b4d00 0x7ff61015af60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff60000bb10 tx=0x7ff60002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fbd50b36d1ea1c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff615326640 1 -- 172.21.15.105:0/2251796105 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60003b070 con 0x7ff6100b4d00 2024-03-08T08:29:54.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff615326640 1 -- 172.21.15.105:0/2251796105 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600036440 con 0x7ff6100b4d00 2024-03-08T08:29:54.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff615326640 1 -- 172.21.15.105:0/2251796105 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600036810 con 0x7ff6100b4d00 2024-03-08T08:29:54.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff61858c640 1 -- 172.21.15.105:0/2251796105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6100b4d00 msgr2=0x7ff61015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff61858c640 1 --2- 172.21.15.105:0/2251796105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6100b4d00 0x7ff61015af60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff60000bb10 tx=0x7ff60002e0c0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff61858c640 1 -- 172.21.15.105:0/2251796105 shutdown_connections 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff61858c640 1 --2- 172.21.15.105:0/2251796105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6100b4d00 0x7ff61015af60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff61858c640 1 --2- 172.21.15.105:0/2251796105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 0x7ff610071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff61858c640 1 -- 172.21.15.105:0/2251796105 >> 172.21.15.105:0/2251796105 conn(0x7ff6100b1740 msgr2=0x7ff6100b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff61858c640 1 -- 172.21.15.105:0/2251796105 shutdown_connections 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.676+0000 7ff61858c640 1 -- 172.21.15.105:0/2251796105 wait complete. 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.677+0000 7ff61858c640 1 Processor -- start 2024-03-08T08:29:54.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff61858c640 1 -- start start 2024-03-08T08:29:54.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff61858c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 0x7ff61014c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff61858c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6100b4d00 0x7ff61014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff61858c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff61015b4a0 0x7ff61014f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff61858c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100729a0 con 0x7ff61015b4a0 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff61858c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610072820 con 0x7ff6100b4d00 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff61858c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61006e7b0 con 0x7ff610072440 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff616b29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff61015b4a0 0x7ff61014f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff616b29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff61015b4a0 0x7ff61014f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44534/0 (socket says 172.21.15.105:44534) 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.678+0000 7ff616b29640 1 -- 172.21.15.105:0/1986068532 learned_addr learned my addr 172.21.15.105:0/1986068532 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff615b27640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6100b4d00 0x7ff61014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff616328640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 0x7ff61014c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff616b29640 1 -- 172.21.15.105:0/1986068532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 msgr2=0x7ff61014c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff616b29640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 0x7ff61014c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff616b29640 1 -- 172.21.15.105:0/1986068532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6100b4d00 msgr2=0x7ff61014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff616b29640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6100b4d00 0x7ff61014f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff616b29640 1 -- 172.21.15.105:0/1986068532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6101f3540 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff616b29640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff61015b4a0 0x7ff61014f610 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b440 tx=0x7ff60c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff606ffd640 1 -- 172.21.15.105:0/1986068532 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c015020 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff606ffd640 1 -- 172.21.15.105:0/1986068532 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c011850 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6101f3820 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff606ffd640 1 -- 172.21.15.105:0/1986068532 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c011c40 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.679+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6101f3de0 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.680+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff610075120 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.680+0000 7ff606ffd640 1 -- 172.21.15.105:0/1986068532 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff60c018020 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.681+0000 7ff606ffd640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff5f406b4d0 0x7ff5f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.681+0000 7ff606ffd640 1 -- 172.21.15.105:0/1986068532 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff60c08a0c0 con 0x7ff61015b4a0 2024-03-08T08:29:54.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.681+0000 7ff616328640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff5f406b4d0 0x7ff5f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.681+0000 7ff616328640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff5f406b4d0 0x7ff5f406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff6101500b0 tx=0x7ff608007310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.685+0000 7ff606ffd640 1 -- 172.21.15.105:0/1986068532 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff60c01f020 con 0x7ff61015b4a0 2024-03-08T08:29:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.856+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff610158880 con 0x7ff61015b4a0 2024-03-08T08:29:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.857+0000 7ff606ffd640 1 -- 172.21.15.105:0/1986068532 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16681 (secure 0 0 0) 0x7ff60c058fd0 con 0x7ff61015b4a0 2024-03-08T08:29:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:29:54.858 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":33,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:28:32.876978+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-08T08:28:29.680862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-08T08:28:30.179255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:29:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff5f406b4d0 msgr2=0x7ff5f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff5f406b4d0 0x7ff5f406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff6101500b0 tx=0x7ff608007310 comp rx=0 tx=0).stop 2024-03-08T08:29:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff61015b4a0 msgr2=0x7ff61014f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff61015b4a0 0x7ff61014f610 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b440 tx=0x7ff60c00b900 comp rx=0 tx=0).stop 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 shutdown_connections 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff5f406b4d0 0x7ff5f406d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff61015b4a0 0x7ff61014f610 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6100b4d00 0x7ff61014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 --2- 172.21.15.105:0/1986068532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff610072440 0x7ff61014c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 >> 172.21.15.105:0/1986068532 conn(0x7ff6100b1740 msgr2=0x7ff61015e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 shutdown_connections 2024-03-08T08:29:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.859+0000 7ff61858c640 1 -- 172.21.15.105:0/1986068532 wait complete. 2024-03-08T08:29:54.871 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-08T08:29:54.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.971+0000 7f51dcb2c640 1 Processor -- start 2024-03-08T08:29:54.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.971+0000 7f51dcb2c640 1 -- start start 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51dcb2c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 0x7f51d8166740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51dcb2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51d806b390 0x7f51d8163fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51dcb2c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d8163850 con 0x7f51d806ded0 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51dcb2c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d8163b50 con 0x7f51d806b390 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51dcb2c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d806fb20 con 0x7f51d81621a0 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51d6d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51d806b390 0x7f51d8163fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51d6d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f51d806b390 0x7f51d8163fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44548/0 (socket says 172.21.15.105:44548) 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51d6d9d640 1 -- 172.21.15.105:0/3323697890 learned_addr learned my addr 172.21.15.105:0/3323697890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51d659c640 1 --2- 172.21.15.105:0/3323697890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 0x7f51d8166740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51d6d9d640 1 -- 172.21.15.105:0/3323697890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 msgr2=0x7f51d8166740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51d6d9d640 1 --2- 172.21.15.105:0/3323697890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 0x7f51d8166740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51d6d9d640 1 -- 172.21.15.105:0/3323697890 >> v1:172.21.15.144:6789/0 conn(0x7f51d806ded0 legacy=0x7f51d806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.972+0000 7f51d6d9d640 1 -- 172.21.15.105:0/3323697890 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d806fe00 con 0x7f51d806b390 2024-03-08T08:29:54.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51d6d9d640 1 --2- 172.21.15.105:0/3323697890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806b390 0x7f51d8163fe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009e50 tx=0x7f51d002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a509bfa9b204d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51d5d9b640 1 -- 172.21.15.105:0/3323697890 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d002cb00 con 0x7f51d806b390 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51d5d9b640 1 -- 172.21.15.105:0/3323697890 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d002cca0 con 0x7f51d806b390 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51d5d9b640 1 -- 172.21.15.105:0/3323697890 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0034610 con 0x7f51d806b390 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51dcb2c640 1 -- 172.21.15.105:0/3323697890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806b390 msgr2=0x7f51d8163fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/3323697890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806b390 0x7f51d8163fe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009e50 tx=0x7f51d002bf60 comp rx=0 tx=0).stop 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51dcb2c640 1 -- 172.21.15.105:0/3323697890 shutdown_connections 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/3323697890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806b390 0x7f51d8163fe0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/3323697890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 0x7f51d8166740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51dcb2c640 1 -- 172.21.15.105:0/3323697890 >> 172.21.15.105:0/3323697890 conn(0x7f51d806bd90 msgr2=0x7f51d806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.973+0000 7f51dcb2c640 1 -- 172.21.15.105:0/3323697890 shutdown_connections 2024-03-08T08:29:54.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.974+0000 7f51dcb2c640 1 -- 172.21.15.105:0/3323697890 wait complete. 2024-03-08T08:29:54.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.975+0000 7f51dcb2c640 1 Processor -- start 2024-03-08T08:29:54.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.975+0000 7f51dcb2c640 1 -- start start 2024-03-08T08:29:54.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.975+0000 7f51dcb2c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d806b390 0x7f51d8148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.975+0000 7f51dcb2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806ded0 0x7f51d814ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.975+0000 7f51dcb2c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 0x7f51d814b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.975+0000 7f51dcb2c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80728a0 con 0x7f51d806ded0 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.975+0000 7f51dcb2c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8163590 con 0x7f51d806b390 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.975+0000 7f51dcb2c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8164520 con 0x7f51d81621a0 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806ded0 0x7f51d814ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806ded0 0x7f51d814ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44550/0 (socket says 172.21.15.105:44550) 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 -- 172.21.15.105:0/1680237224 learned_addr learned my addr 172.21.15.105:0/1680237224 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51d6d9d640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 0x7f51d814b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51d659c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d806b390 0x7f51d8148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 -- 172.21.15.105:0/1680237224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 msgr2=0x7f51d814b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 0x7f51d814b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 -- 172.21.15.105:0/1680237224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d806b390 msgr2=0x7f51d8148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d806b390 0x7f51d8148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 -- 172.21.15.105:0/1680237224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d814b880 con 0x7f51d806ded0 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51cffff640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806ded0 0x7f51d814ad30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c400 tx=0x7f51c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51ceffd640 1 -- 172.21.15.105:0/1680237224 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c00052d0 con 0x7f51d806ded0 2024-03-08T08:29:54.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51ceffd640 1 -- 172.21.15.105:0/1680237224 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c0005490 con 0x7f51d806ded0 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d814bb60 con 0x7f51d806ded0 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51ceffd640 1 -- 172.21.15.105:0/1680237224 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0007260 con 0x7f51d806ded0 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.976+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d8159020 con 0x7f51d806ded0 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.977+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d814bd30 con 0x7f51d806ded0 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.978+0000 7f51ceffd640 1 -- 172.21.15.105:0/1680237224 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51c0007400 con 0x7f51d806ded0 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.978+0000 7f51ceffd640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51b806b500 0x7f51b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.978+0000 7f51ceffd640 1 -- 172.21.15.105:0/1680237224 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51c0089650 con 0x7f51d806ded0 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.978+0000 7f51d659c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51b806b500 0x7f51b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:54.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.978+0000 7f51d659c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51b806b500 0x7f51b806d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009a30 tx=0x7f51c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:54.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:54.982+0000 7f51ceffd640 1 -- 172.21.15.105:0/1680237224 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51d814bd30 con 0x7f51d806ded0 2024-03-08T08:29:55.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.188+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f51d8075120 con 0x7f51d806ded0 2024-03-08T08:29:55.189 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-03-08T08:29:55.188+0000 7f935598c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-08T08:29:55.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.190+0000 7f51ceffd640 1 -- 172.21.15.105:0/1680237224 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) v1 ==== 161+0+0 (secure 0 0 0) 0x7f51c0058560 con 0x7f51d806ded0 2024-03-08T08:29:55.191 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.192+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51b806b500 msgr2=0x7f51b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.192+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51b806b500 0x7f51b806d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009a30 tx=0x7f51c8008040 comp rx=0 tx=0).stop 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.192+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806ded0 msgr2=0x7f51d814ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.192+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806ded0 0x7f51d814ad30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c400 tx=0x7f51c000c8c0 comp rx=0 tx=0).stop 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.193+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 shutdown_connections 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.193+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51b806b500 0x7f51b806d9c0 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009a30 tx=0x7f51c8008040 comp rx=0 tx=0).stop 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.193+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51d81621a0 0x7f51d814b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.193+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51d806ded0 0x7f51d814ad30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:55.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.193+0000 7f51dcb2c640 1 --2- 172.21.15.105:0/1680237224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51d806b390 0x7f51d8148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:55.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.193+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 >> 172.21.15.105:0/1680237224 conn(0x7f51d806bd90 msgr2=0x7f51d806e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:55.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.193+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 shutdown_connections 2024-03-08T08:29:55.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.193+0000 7f51dcb2c640 1 -- 172.21.15.105:0/1680237224 wait complete. 2024-03-08T08:29:55.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-08T08:29:55.231 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-03-08T08:29:55.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.308+0000 7f82373d7640 1 Processor -- start 2024-03-08T08:29:55.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.308+0000 7f82373d7640 1 -- start start 2024-03-08T08:29:55.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.309+0000 7f82373d7640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f82300b4d00 0x7f823015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:55.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.309+0000 7f82373d7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823015b940 con 0x7f823015b4a0 2024-03-08T08:29:55.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.309+0000 7f82373d7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823015bc20 con 0x7f8230072440 2024-03-08T08:29:55.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.309+0000 7f82373d7640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8230070840 con 0x7f82300b4d00 2024-03-08T08:29:55.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.309+0000 7f8235974640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f823015b4a0 0x7f823006e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51806/0 (socket says 172.21.15.105:51806) 2024-03-08T08:29:55.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.309+0000 7f8235974640 1 -- 172.21.15.105:0/2373602322 learned_addr learned my addr 172.21.15.105:0/2373602322 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:55.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.309+0000 7f8234972640 1 --2- 172.21.15.105:0/2373602322 >> v2:172.21.15.144:3300/0 conn(0x7f82300b4d00 0x7f823015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:55.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.310+0000 7f8227fff640 1 -- 172.21.15.105:0/2373602322 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765128907 0 0) 0x7f823015b940 con 0x7f823015b4a0 2024-03-08T08:29:55.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.310+0000 7f8227fff640 1 -- 172.21.15.105:0/2373602322 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f821c003520 con 0x7f823015b4a0 2024-03-08T08:29:55.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.310+0000 7f8227fff640 1 -- 172.21.15.105:0/2373602322 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491505608 0 0) 0x7f823015bc20 con 0x7f8230072440 2024-03-08T08:29:55.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.310+0000 7f8227fff640 1 -- 172.21.15.105:0/2373602322 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f823015b940 con 0x7f8230072440 2024-03-08T08:29:55.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.310+0000 7f8227fff640 1 -- 172.21.15.105:0/2373602322 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2062566407 0 0) 0x7f821c003520 con 0x7f823015b4a0 2024-03-08T08:29:55.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.310+0000 7f8234972640 1 -- 172.21.15.105:0/2373602322 >> v1:172.21.15.144:6789/0 conn(0x7f8230072440 legacy=0x7f8230071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:55.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.310+0000 7f8234972640 1 -- 172.21.15.105:0/2373602322 >> v1:172.21.15.105:6789/0 conn(0x7f823015b4a0 legacy=0x7f823006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:55.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.310+0000 7f8234972640 1 -- 172.21.15.105:0/2373602322 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8230070b20 con 0x7f82300b4d00 2024-03-08T08:29:55.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f8234972640 1 --2- 172.21.15.105:0/2373602322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82300b4d00 0x7f823015af60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82200097a0 tx=0x7f822002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=243e74c46fcef6c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f8227fff640 1 -- 172.21.15.105:0/2373602322 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220038070 con 0x7f82300b4d00 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f8227fff640 1 -- 172.21.15.105:0/2373602322 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8220031500 con 0x7f82300b4d00 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f8227fff640 1 -- 172.21.15.105:0/2373602322 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228001670 con 0x7f82300b4d00 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f82373d7640 1 -- 172.21.15.105:0/2373602322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82300b4d00 msgr2=0x7f823015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f82373d7640 1 --2- 172.21.15.105:0/2373602322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82300b4d00 0x7f823015af60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82200097a0 tx=0x7f822002b820 comp rx=0 tx=0).stop 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f82373d7640 1 -- 172.21.15.105:0/2373602322 shutdown_connections 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f82373d7640 1 --2- 172.21.15.105:0/2373602322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82300b4d00 0x7f823015af60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f82373d7640 1 -- 172.21.15.105:0/2373602322 >> 172.21.15.105:0/2373602322 conn(0x7f82300b1740 msgr2=0x7f82300b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f82373d7640 1 -- 172.21.15.105:0/2373602322 shutdown_connections 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.311+0000 7f82373d7640 1 -- 172.21.15.105:0/2373602322 wait complete. 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.312+0000 7f82373d7640 1 Processor -- start 2024-03-08T08:29:55.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.312+0000 7f82373d7640 1 -- start start 2024-03-08T08:29:55.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f82373d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8230072440 0x7f82301433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:55.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f82373d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82300b4d00 0x7f8230154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:55.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f82373d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f823015b4a0 0x7f8230159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:55.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f82373d7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823006e930 con 0x7f8230072440 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f82373d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823006e7b0 con 0x7f82300b4d00 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f82373d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82300cdd10 con 0x7f823015b4a0 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f8235173640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8230072440 0x7f82301433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f8235173640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8230072440 0x7f82301433b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44578/0 (socket says 172.21.15.105:44578) 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.313+0000 7f8235173640 1 -- 172.21.15.105:0/2026385281 learned_addr learned my addr 172.21.15.105:0/2026385281 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8235173640 1 -- 172.21.15.105:0/2026385281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f823015b4a0 msgr2=0x7f8230159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8235173640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f823015b4a0 0x7f8230159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8235173640 1 -- 172.21.15.105:0/2026385281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82300b4d00 msgr2=0x7f8230154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8235173640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82300b4d00 0x7f8230154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8235173640 1 -- 172.21.15.105:0/2026385281 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8230143c70 con 0x7f8230072440 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8235173640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8230072440 0x7f82301433b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c830 tx=0x7f822c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8225ffb640 1 -- 172.21.15.105:0/2026385281 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c003b70 con 0x7f8230072440 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8225ffb640 1 -- 172.21.15.105:0/2026385281 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c003d10 con 0x7f8230072440 2024-03-08T08:29:55.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8230143f50 con 0x7f8230072440 2024-03-08T08:29:55.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f8225ffb640 1 -- 172.21.15.105:0/2026385281 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00aa90 con 0x7f8230072440 2024-03-08T08:29:55.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.314+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8230151860 con 0x7f8230072440 2024-03-08T08:29:55.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.315+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8230071d20 con 0x7f8230072440 2024-03-08T08:29:55.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.315+0000 7f8225ffb640 1 -- 172.21.15.105:0/2026385281 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f822c00ac30 con 0x7f8230072440 2024-03-08T08:29:55.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.316+0000 7f8225ffb640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f821406b4d0 0x7f821406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:55.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.316+0000 7f8234972640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f821406b4d0 0x7f821406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:55.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.316+0000 7f8225ffb640 1 -- 172.21.15.105:0/2026385281 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f822c08a350 con 0x7f8230072440 2024-03-08T08:29:55.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.316+0000 7f8234972640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f821406b4d0 0x7f821406d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8220035000 tx=0x7f8220030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:55.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.320+0000 7f8225ffb640 1 -- 172.21.15.105:0/2026385281 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8230071d20 con 0x7f8230072440 2024-03-08T08:29:55.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:55.527+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8230075120 con 0x7f8230072440 2024-03-08T08:29:56.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.236+0000 7f8225ffb640 1 -- 172.21.15.105:0/2026385281 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f822c0591b0 con 0x7f8230072440 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f821406b4d0 msgr2=0x7f821406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f821406b4d0 0x7f821406d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8220035000 tx=0x7f8220030000 comp rx=0 tx=0).stop 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8230072440 msgr2=0x7f82301433b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8230072440 0x7f82301433b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c830 tx=0x7f822c00ccf0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 shutdown_connections 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f821406b4d0 0x7f821406d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f823015b4a0 0x7f8230159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82300b4d00 0x7f8230154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 --2- 172.21.15.105:0/2026385281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8230072440 0x7f82301433b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 >> 172.21.15.105:0/2026385281 conn(0x7f82300b1740 msgr2=0x7f823006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:56.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 shutdown_connections 2024-03-08T08:29:56.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.240+0000 7f82373d7640 1 -- 172.21.15.105:0/2026385281 wait complete. 2024-03-08T08:29:56.254 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-08T08:29:56.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.403+0000 7fb79b62a640 1 Processor -- start 2024-03-08T08:29:56.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb79b62a640 1 -- start start 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb79b62a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb79415a2f0 0x7fb7940cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb79b62a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb79406dfe0 0x7fb79406e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb79b62a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79415a6d0 con 0x7fb794071900 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb79b62a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7940cdc30 con 0x7fb79406dfe0 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb79b62a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940cf740 con 0x7fb79415a2f0 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb799bc7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb79406dfe0 0x7fb79406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb798bc5640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb794071900 0x7fb79415e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51820/0 (socket says 172.21.15.105:51820) 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.404+0000 7fb798bc5640 1 -- 172.21.15.105:0/2267145679 learned_addr learned my addr 172.21.15.105:0/2267145679 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.405+0000 7fb7993c6640 1 --2- 172.21.15.105:0/2267145679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb79415a2f0 0x7fb7940cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.405+0000 7fb799bc7640 1 -- 172.21.15.105:0/2267145679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb79415a2f0 msgr2=0x7fb7940cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:56.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.405+0000 7fb799bc7640 1 --2- 172.21.15.105:0/2267145679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb79415a2f0 0x7fb7940cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.405+0000 7fb799bc7640 1 -- 172.21.15.105:0/2267145679 >> v1:172.21.15.105:6789/0 conn(0x7fb794071900 legacy=0x7fb79415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:56.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.405+0000 7fb78bfff640 1 -- 172.21.15.105:0/2267145679 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397858354 0 0) 0x7fb79415a6d0 con 0x7fb794071900 2024-03-08T08:29:56.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.405+0000 7fb799bc7640 1 -- 172.21.15.105:0/2267145679 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794071dc0 con 0x7fb79406dfe0 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.405+0000 7fb799bc7640 1 --2- 172.21.15.105:0/2267145679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79406dfe0 0x7fb79406e470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d90 tx=0x7fb78c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc893b5b594aa0a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb78bfff640 1 -- 172.21.15.105:0/2267145679 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c039070 con 0x7fb79406dfe0 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb78bfff640 1 -- 172.21.15.105:0/2267145679 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c02cc50 con 0x7fb79406dfe0 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb78bfff640 1 -- 172.21.15.105:0/2267145679 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c034440 con 0x7fb79406dfe0 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb79b62a640 1 -- 172.21.15.105:0/2267145679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79406dfe0 msgr2=0x7fb79406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb79b62a640 1 --2- 172.21.15.105:0/2267145679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79406dfe0 0x7fb79406e470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d90 tx=0x7fb78c02bea0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb79b62a640 1 -- 172.21.15.105:0/2267145679 shutdown_connections 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb79b62a640 1 --2- 172.21.15.105:0/2267145679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79406dfe0 0x7fb79406e470 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb79b62a640 1 --2- 172.21.15.105:0/2267145679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb79415a2f0 0x7fb7940cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb79b62a640 1 -- 172.21.15.105:0/2267145679 >> 172.21.15.105:0/2267145679 conn(0x7fb7940b17b0 msgr2=0x7fb7940b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:56.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb79b62a640 1 -- 172.21.15.105:0/2267145679 shutdown_connections 2024-03-08T08:29:56.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.406+0000 7fb79b62a640 1 -- 172.21.15.105:0/2267145679 wait complete. 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.407+0000 7fb79b62a640 1 Processor -- start 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb79b62a640 1 -- start start 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb79b62a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb79406dfe0 0x7fb7941590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb79b62a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb794071900 0x7fb794157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb79b62a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79415a2f0 0x7fb79415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb79b62a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb794072660 con 0x7fb79415a2f0 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb79b62a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb794072380 con 0x7fb79406dfe0 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb79b62a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79406d750 con 0x7fb794071900 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb799bc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79415a2f0 0x7fb79415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb799bc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79415a2f0 0x7fb79415cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44608/0 (socket says 172.21.15.105:44608) 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.408+0000 7fb799bc7640 1 -- 172.21.15.105:0/1477080389 learned_addr learned my addr 172.21.15.105:0/1477080389 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb799bc7640 1 -- 172.21.15.105:0/1477080389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb794071900 msgr2=0x7fb794157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb799bc7640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb794071900 0x7fb794157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb799bc7640 1 -- 172.21.15.105:0/1477080389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb79406dfe0 msgr2=0x7fb7941590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb799bc7640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb79406dfe0 0x7fb7941590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb799bc7640 1 -- 172.21.15.105:0/1477080389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794158050 con 0x7fb79415a2f0 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb799bc7640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79415a2f0 0x7fb79415cb80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d90 tx=0x7fb78c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb789ffb640 1 -- 172.21.15.105:0/1477080389 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c044020 con 0x7fb79415a2f0 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb789ffb640 1 -- 172.21.15.105:0/1477080389 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c03a040 con 0x7fb79415a2f0 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb794158330 con 0x7fb79415a2f0 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb789ffb640 1 -- 172.21.15.105:0/1477080389 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c039050 con 0x7fb79415a2f0 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.409+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb794158bd0 con 0x7fb79415a2f0 2024-03-08T08:29:56.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.410+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb794158720 con 0x7fb79415a2f0 2024-03-08T08:29:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.410+0000 7fb789ffb640 1 -- 172.21.15.105:0/1477080389 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb78c003990 con 0x7fb79415a2f0 2024-03-08T08:29:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.411+0000 7fb789ffb640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb77806b4d0 0x7fb77806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.411+0000 7fb789ffb640 1 -- 172.21.15.105:0/1477080389 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb78c0ac690 con 0x7fb79415a2f0 2024-03-08T08:29:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.411+0000 7fb7993c6640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb77806b4d0 0x7fb77806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:56.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.411+0000 7fb7993c6640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb77806b4d0 0x7fb77806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb790006a90 tx=0x7fb790006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:56.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.415+0000 7fb789ffb640 1 -- 172.21.15.105:0/1477080389 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb78c04e060 con 0x7fb79415a2f0 2024-03-08T08:29:56.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:56.585+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb794075200 con 0x7fb79415a2f0 2024-03-08T08:29:57.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.245+0000 7fb789ffb640 1 -- 172.21.15.105:0/1477080389 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb78c07b540 con 0x7fb79415a2f0 2024-03-08T08:29:57.246 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-03-08T08:29:57.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.248+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb77806b4d0 msgr2=0x7fb77806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:57.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.248+0000 7fb79b62a640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb77806b4d0 0x7fb77806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb790006a90 tx=0x7fb790006a20 comp rx=0 tx=0).stop 2024-03-08T08:29:57.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.248+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79415a2f0 msgr2=0x7fb79415cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:57.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.248+0000 7fb79b62a640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79415a2f0 0x7fb79415cb80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009d90 tx=0x7fb78c004a40 comp rx=0 tx=0).stop 2024-03-08T08:29:57.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.249+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 shutdown_connections 2024-03-08T08:29:57.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.249+0000 7fb79b62a640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb77806b4d0 0x7fb77806d990 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb790006a90 tx=0x7fb790006a20 comp rx=0 tx=0).stop 2024-03-08T08:29:57.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.249+0000 7fb79b62a640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb79415a2f0 0x7fb79415cb80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.249+0000 7fb79b62a640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb794071900 0x7fb794157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.249+0000 7fb79b62a640 1 --2- 172.21.15.105:0/1477080389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb79406dfe0 0x7fb7941590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.249+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 >> 172.21.15.105:0/1477080389 conn(0x7fb7940b17b0 msgr2=0x7fb7940cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:57.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.249+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 shutdown_connections 2024-03-08T08:29:57.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.249+0000 7fb79b62a640 1 -- 172.21.15.105:0/1477080389 wait complete. 2024-03-08T08:29:57.263 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-08T08:29:57.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.369+0000 7f386e7c7640 1 Processor -- start 2024-03-08T08:29:57.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.369+0000 7f386e7c7640 1 -- start start 2024-03-08T08:29:57.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f386e7c7640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3868071900 0x7f38680cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:57.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f386e7c7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386815a6d0 con 0x7f386815a2f0 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f386e7c7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3868071dc0 con 0x7f386806dfe0 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f386e7c7640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38680cdc30 con 0x7f3868071900 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3867fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f386815a2f0 0x7f386815e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51830/0 (socket says 172.21.15.105:51830) 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3867fff640 1 -- 172.21.15.105:0/109765301 learned_addr learned my addr 172.21.15.105:0/109765301 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446109260 0 0) 0x7f386815a6d0 con 0x7f386815a2f0 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3850003520 con 0x7f386815a2f0 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f38677fe640 1 --2- 172.21.15.105:0/109765301 >> v2:172.21.15.144:3300/0 conn(0x7f3868071900 0x7f38680cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 36936979 0 0) 0x7f3850003520 con 0x7f386815a2f0 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 >> v2:172.21.15.144:3300/0 conn(0x7f3868071900 msgr2=0x7f38680cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 --2- 172.21.15.105:0/109765301 >> v2:172.21.15.144:3300/0 conn(0x7f3868071900 0x7f38680cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 >> v1:172.21.15.144:6789/0 conn(0x7f386806dfe0 legacy=0x7f38680b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680720a0 con 0x7f386815a2f0 2024-03-08T08:29:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f3858001670 con 0x7f386815a2f0 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 >> v1:172.21.15.105:6789/0 conn(0x7f386815a2f0 legacy=0x7f386815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38500039b0 0x7f3850003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850004800 0x7f3850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38500051c0 0x7f385000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38680cdc30 con 0x7f3850004800 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385000c060 con 0x7f38500051c0 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385000c260 con 0x7f38500039b0 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.370+0000 7f38677fe640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850004800 0x7f3850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f386cd64640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38500051c0 0x7f385000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f3867fff640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38500039b0 0x7f3850003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f38677fe640 1 -- 172.21.15.105:0/109765301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38500039b0 msgr2=0x7f3850003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f38677fe640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38500039b0 0x7f3850003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f38677fe640 1 -- 172.21.15.105:0/109765301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38500051c0 msgr2=0x7f385000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f38677fe640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38500051c0 0x7f385000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f386cd64640 1 -- 172.21.15.105:0/109765301 reap_dead start 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f38677fe640 1 -- 172.21.15.105:0/109765301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3850003520 con 0x7f3850004800 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.371+0000 7f38677fe640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850004800 0x7f3850004c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f385c00aac0 tx=0x7f3850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739b4c5ce2f968b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00b040 con 0x7f3850004800 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c034600 con 0x7f3850004800 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f3866ffd640 1 -- 172.21.15.105:0/109765301 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0349d0 con 0x7f3850004800 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f386e7c7640 1 -- 172.21.15.105:0/109765301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850004800 msgr2=0x7f3850004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f386e7c7640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850004800 0x7f3850004c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f385c00aac0 tx=0x7f3850005830 comp rx=0 tx=0).stop 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f386e7c7640 1 -- 172.21.15.105:0/109765301 shutdown_connections 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f386e7c7640 1 --2- 172.21.15.105:0/109765301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3850004800 0x7f3850004c80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f386e7c7640 1 -- 172.21.15.105:0/109765301 >> 172.21.15.105:0/109765301 conn(0x7f38680b17b0 msgr2=0x7f38680b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f386e7c7640 1 -- 172.21.15.105:0/109765301 shutdown_connections 2024-03-08T08:29:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.372+0000 7f386e7c7640 1 -- 172.21.15.105:0/109765301 wait complete. 2024-03-08T08:29:57.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.373+0000 7f386e7c7640 1 Processor -- start 2024-03-08T08:29:57.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f386e7c7640 1 -- start start 2024-03-08T08:29:57.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f386e7c7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3850004800 0x7f3868141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:57.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f386e7c7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681450a0 0x7f38680c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:57.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f386e7c7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38680c9fd0 0x7f38680c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:57.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f386e7c7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3868072660 con 0x7f38681450a0 2024-03-08T08:29:57.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f386e7c7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38680cf740 con 0x7f3850004800 2024-03-08T08:29:57.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f386e7c7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38680cd6f0 con 0x7f38680c9fd0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f38677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681450a0 0x7f38680c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f38677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681450a0 0x7f38680c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44616/0 (socket says 172.21.15.105:44616) 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f38677fe640 1 -- 172.21.15.105:0/1825174696 learned_addr learned my addr 172.21.15.105:0/1825174696 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f3867fff640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3850004800 0x7f3868141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.374+0000 7f386cd64640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38680c9fd0 0x7f38680c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f38677fe640 1 -- 172.21.15.105:0/1825174696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38680c9fd0 msgr2=0x7f38680c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f38677fe640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38680c9fd0 0x7f38680c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f38677fe640 1 -- 172.21.15.105:0/1825174696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3850004800 msgr2=0x7f3868141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f38677fe640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3850004800 0x7f3868141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f38677fe640 1 -- 172.21.15.105:0/1825174696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38681ef3d0 con 0x7f38681450a0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f38677fe640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681450a0 0x7f38680c9a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f385c000c00 tx=0x7f385c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f3864ff9640 1 -- 172.21.15.105:0/1825174696 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00b040 con 0x7f38681450a0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f3864ff9640 1 -- 172.21.15.105:0/1825174696 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c036e10 con 0x7f38681450a0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f3864ff9640 1 -- 172.21.15.105:0/1825174696 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c036290 con 0x7f38681450a0 2024-03-08T08:29:57.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3868142980 con 0x7f38681450a0 2024-03-08T08:29:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.375+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3868142f40 con 0x7f38681450a0 2024-03-08T08:29:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.376+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3868075200 con 0x7f38681450a0 2024-03-08T08:29:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.376+0000 7f3864ff9640 1 -- 172.21.15.105:0/1825174696 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f385c036430 con 0x7f38681450a0 2024-03-08T08:29:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.377+0000 7f3864ff9640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f385406b500 0x7f385406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.377+0000 7f3867fff640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f385406b500 0x7f385406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.377+0000 7f3864ff9640 1 -- 172.21.15.105:0/1825174696 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f385c0adfa0 con 0x7f38681450a0 2024-03-08T08:29:57.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.377+0000 7f3867fff640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f385406b500 0x7f385406d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3858001050 tx=0x7f3858000f80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:57.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.381+0000 7f3864ff9640 1 -- 172.21.15.105:0/1825174696 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38681ef3d0 con 0x7f38681450a0 2024-03-08T08:29:57.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:57.553+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f386806d1f0 con 0x7f38681450a0 2024-03-08T08:29:58.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.251+0000 7f3864ff9640 1 -- 172.21.15.105:0/1825174696 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f385c042030 con 0x7f38681450a0 2024-03-08T08:29:58.252 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-03-08T08:29:58.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f385406b500 msgr2=0x7f385406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f385406b500 0x7f385406d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3858001050 tx=0x7f3858000f80 comp rx=0 tx=0).stop 2024-03-08T08:29:58.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681450a0 msgr2=0x7f38680c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681450a0 0x7f38680c9a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f385c000c00 tx=0x7f385c004730 comp rx=0 tx=0).stop 2024-03-08T08:29:58.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 shutdown_connections 2024-03-08T08:29:58.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f385406b500 0x7f385406d9c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38680c9fd0 0x7f38680c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681450a0 0x7f38680c9a90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 --2- 172.21.15.105:0/1825174696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3850004800 0x7f3868141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 >> 172.21.15.105:0/1825174696 conn(0x7f38680b17b0 msgr2=0x7f386815b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:58.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 shutdown_connections 2024-03-08T08:29:58.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.255+0000 7f386e7c7640 1 -- 172.21.15.105:0/1825174696 wait complete. 2024-03-08T08:29:58.271 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-08T08:29:58.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.380+0000 7fe21248f640 1 Processor -- start 2024-03-08T08:29:58.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.380+0000 7fe21248f640 1 -- start start 2024-03-08T08:29:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.380+0000 7fe21248f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 0x7fe20c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.380+0000 7fe21248f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe20c0cfb40 0x7fe20c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.380+0000 7fe21248f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20c0b29e0 con 0x7fe20c161520 2024-03-08T08:29:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.380+0000 7fe21248f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c0d1930 con 0x7fe20c0cfb40 2024-03-08T08:29:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.380+0000 7fe21248f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c0cecd0 con 0x7fe20c0ce330 2024-03-08T08:29:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.381+0000 7fe210a2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe20c0cfb40 0x7fe20c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.381+0000 7fe210a2c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe20c0cfb40 0x7fe20c0cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44622/0 (socket says 172.21.15.105:44622) 2024-03-08T08:29:58.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.381+0000 7fe210a2c640 1 -- 172.21.15.105:0/2409479104 learned_addr learned my addr 172.21.15.105:0/2409479104 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:58.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.381+0000 7fe210a2c640 1 -- 172.21.15.105:0/2409479104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 msgr2=0x7fe20c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:58.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.381+0000 7fe210a2c640 1 --2- 172.21.15.105:0/2409479104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 0x7fe20c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.381+0000 7fe210a2c640 1 -- 172.21.15.105:0/2409479104 >> v1:172.21.15.144:6789/0 conn(0x7fe20c161520 legacy=0x7fe20c0b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:58.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.381+0000 7fe210a2c640 1 -- 172.21.15.105:0/2409479104 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c0d1650 con 0x7fe20c0cfb40 2024-03-08T08:29:58.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe210a2c640 1 --2- 172.21.15.105:0/2409479104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c0cfb40 0x7fe20c0cffd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe204009e50 tx=0x7fe20402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac9450b3c843b8c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:58.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe20affd640 1 -- 172.21.15.105:0/2409479104 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20402ca70 con 0x7fe20c0cfb40 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe20affd640 1 -- 172.21.15.105:0/2409479104 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20402cc10 con 0x7fe20c0cfb40 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe20affd640 1 -- 172.21.15.105:0/2409479104 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040343f0 con 0x7fe20c0cfb40 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe21248f640 1 -- 172.21.15.105:0/2409479104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c0cfb40 msgr2=0x7fe20c0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe21248f640 1 --2- 172.21.15.105:0/2409479104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c0cfb40 0x7fe20c0cffd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe204009e50 tx=0x7fe20402bed0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe21248f640 1 -- 172.21.15.105:0/2409479104 shutdown_connections 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe21248f640 1 --2- 172.21.15.105:0/2409479104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c0cfb40 0x7fe20c0cffd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe21248f640 1 --2- 172.21.15.105:0/2409479104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 0x7fe20c0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe21248f640 1 -- 172.21.15.105:0/2409479104 >> 172.21.15.105:0/2409479104 conn(0x7fe20c0b36a0 msgr2=0x7fe20c0b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe21248f640 1 -- 172.21.15.105:0/2409479104 shutdown_connections 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.382+0000 7fe21248f640 1 -- 172.21.15.105:0/2409479104 wait complete. 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.383+0000 7fe21248f640 1 Processor -- start 2024-03-08T08:29:58.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.384+0000 7fe21248f640 1 -- start start 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.384+0000 7fe21248f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 0x7fe20c0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.384+0000 7fe21248f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe20c0cfb40 0x7fe20c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.384+0000 7fe21248f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c161520 0x7fe20c1f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.384+0000 7fe21248f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c06ca40 con 0x7fe20c161520 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.384+0000 7fe21248f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c06c770 con 0x7fe20c0cfb40 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.384+0000 7fe21248f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c06ced0 con 0x7fe20c0ce330 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.384+0000 7fe210a2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c161520 0x7fe20c1f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe210a2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c161520 0x7fe20c1f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44628/0 (socket says 172.21.15.105:44628) 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe210a2c640 1 -- 172.21.15.105:0/405679698 learned_addr learned my addr 172.21.15.105:0/405679698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe20b7fe640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe20c0cfb40 0x7fe20c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe20bfff640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 0x7fe20c0c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe210a2c640 1 -- 172.21.15.105:0/405679698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 msgr2=0x7fe20c0c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe210a2c640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 0x7fe20c0c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe210a2c640 1 -- 172.21.15.105:0/405679698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe20c0cfb40 msgr2=0x7fe20c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe210a2c640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe20c0cfb40 0x7fe20c0c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe210a2c640 1 -- 172.21.15.105:0/405679698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c0c6660 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe210a2c640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c161520 0x7fe20c1f3e20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe204036040 tx=0x7fe204004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe208ff9640 1 -- 172.21.15.105:0/405679698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204043020 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe208ff9640 1 -- 172.21.15.105:0/405679698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20403e450 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe208ff9640 1 -- 172.21.15.105:0/405679698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20403e860 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c0c6940 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.385+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c1f4360 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.386+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20c0c6e60 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.386+0000 7fe208ff9640 1 -- 172.21.15.105:0/405679698 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe204046020 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.387+0000 7fe208ff9640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1f806b4d0 0x7fe1f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.387+0000 7fe208ff9640 1 -- 172.21.15.105:0/405679698 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe2040ac790 con 0x7fe20c161520 2024-03-08T08:29:58.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.387+0000 7fe20bfff640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1f806b4d0 0x7fe1f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.387+0000 7fe20bfff640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1f806b4d0 0x7fe1f806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00a730 tx=0x7fe1fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:58.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.391+0000 7fe208ff9640 1 -- 172.21.15.105:0/405679698 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe20c0c6e60 con 0x7fe20c161520 2024-03-08T08:29:58.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.572+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe20c075120 con 0x7fe20c161520 2024-03-08T08:29:58.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.575+0000 7fe208ff9640 1 -- 172.21.15.105:0/405679698 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe20404d030 con 0x7fe20c161520 2024-03-08T08:29:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1f806b4d0 msgr2=0x7fe1f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1f806b4d0 0x7fe1f806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00a730 tx=0x7fe1fc008040 comp rx=0 tx=0).stop 2024-03-08T08:29:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c161520 msgr2=0x7fe20c1f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c161520 0x7fe20c1f3e20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe204036040 tx=0x7fe204004a40 comp rx=0 tx=0).stop 2024-03-08T08:29:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 shutdown_connections 2024-03-08T08:29:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe1f806b4d0 0x7fe1f806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe20c161520 0x7fe20c1f3e20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe20c0cfb40 0x7fe20c0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 --2- 172.21.15.105:0/405679698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe20c0ce330 0x7fe20c0c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 >> 172.21.15.105:0/405679698 conn(0x7fe20c0b36a0 msgr2=0x7fe20c0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 shutdown_connections 2024-03-08T08:29:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.578+0000 7fe21248f640 1 -- 172.21.15.105:0/405679698 wait complete. 2024-03-08T08:29:58.590 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters' 2024-03-08T08:29:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.688+0000 7f80a5c47640 1 Processor -- start 2024-03-08T08:29:58.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.688+0000 7f80a5c47640 1 -- start start 2024-03-08T08:29:58.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f80a5c47640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a01469e0 0x7f80a006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f80a5c47640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a00adcd0 con 0x7f80a00ad870 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f80a5c47640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a006deb0 con 0x7f80a006ac80 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f80a5c47640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a006e190 con 0x7f80a01469e0 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809effd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f80a00ad870 0x7f80a006d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51844/0 (socket says 172.21.15.105:51844) 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809effd640 1 -- 172.21.15.105:0/562431120 learned_addr learned my addr 172.21.15.105:0/562431120 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894092062 0 0) 0x7f80a00adcd0 con 0x7f80a00ad870 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8088003520 con 0x7f80a00ad870 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809f7fe640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a01469e0 0x7f80a006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2174207419 0 0) 0x7f8088003520 con 0x7f80a00ad870 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a01469e0 msgr2=0x7f80a006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a01469e0 0x7f80a006fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 >> v1:172.21.15.144:6789/0 conn(0x7f80a006ac80 legacy=0x7f80a00b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00b42f0 con 0x7f80a00ad870 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f8094001670 con 0x7f80a00ad870 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 >> v1:172.21.15.105:6789/0 conn(0x7f80a00ad870 legacy=0x7f80a006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a01469e0 0x7f80880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088003f60 0x7f8088004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8088004d40 0x7f80880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a006e190 con 0x7f8088003f60 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808800bb50 con 0x7f8088004d40 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.689+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808800bd50 con 0x7f80a01469e0 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.690+0000 7f809effd640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088003f60 0x7f8088004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.690+0000 7f809f7fe640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a01469e0 0x7f80880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.690+0000 7f809ffff640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8088004d40 0x7f80880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.690+0000 7f809effd640 1 -- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a01469e0 msgr2=0x7f80880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.690+0000 7f809effd640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a01469e0 0x7f80880039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.690+0000 7f809effd640 1 -- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8088004d40 msgr2=0x7f80880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.690+0000 7f809effd640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8088004d40 0x7f80880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.690+0000 7f809effd640 1 -- 172.21.15.105:0/562431120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8088003520 con 0x7f8088003f60 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f809effd640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088003f60 0x7f8088004800 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f809400c240 tx=0x7f8088005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b5183225bb407b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:58.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094007cf0 con 0x7f8088003f60 2024-03-08T08:29:58.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8094030070 con 0x7f8088003f60 2024-03-08T08:29:58.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f809e7fc640 1 -- 172.21.15.105:0/562431120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094005d10 con 0x7f8088003f60 2024-03-08T08:29:58.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f80a5c47640 1 -- 172.21.15.105:0/562431120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088003f60 msgr2=0x7f8088004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f80a5c47640 1 --2- 172.21.15.105:0/562431120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088003f60 0x7f8088004800 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f809400c240 tx=0x7f8088005830 comp rx=0 tx=0).stop 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f809ffff640 1 -- 172.21.15.105:0/562431120 reap_dead start 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f80a5c47640 1 -- 172.21.15.105:0/562431120 shutdown_connections 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f80a5c47640 1 -- 172.21.15.105:0/562431120 >> 172.21.15.105:0/562431120 conn(0x7f80a00b2670 msgr2=0x7f80a00b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f80a5c47640 1 -- 172.21.15.105:0/562431120 shutdown_connections 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.691+0000 7f80a5c47640 1 -- 172.21.15.105:0/562431120 wait complete. 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.692+0000 7f80a5c47640 1 Processor -- start 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f80a5c47640 1 -- start start 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f80a5c47640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8088003f60 0x7f80a00af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f80a5c47640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015d3c0 0x7f80a015d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f80a5c47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80a015f520 0x7f80a015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f80a5c47640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a0146dc0 con 0x7f80a015f520 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f80a5c47640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a006e470 con 0x7f80a015d3c0 2024-03-08T08:29:58.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f80a5c47640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a006fd60 con 0x7f8088003f60 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f809ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80a015f520 0x7f80a015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f809ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80a015f520 0x7f80a015f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44652/0 (socket says 172.21.15.105:44652) 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f809ffff640 1 -- 172.21.15.105:0/3950651586 learned_addr learned my addr 172.21.15.105:0/3950651586 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f809effd640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015d3c0 0x7f80a015d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.693+0000 7f809f7fe640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8088003f60 0x7f80a00af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f809ffff640 1 -- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8088003f60 msgr2=0x7f80a00af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f809ffff640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8088003f60 0x7f80a00af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f809ffff640 1 -- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015d3c0 msgr2=0x7f80a015d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f809ffff640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015d3c0 0x7f80a015d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f809ffff640 1 -- 172.21.15.105:0/3950651586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a015dd80 con 0x7f80a015f520 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f809f7fe640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8088003f60 0x7f80a00af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f809effd640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015d3c0 0x7f80a015d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f809ffff640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80a015f520 0x7f80a015f9a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f80a00b6a90 tx=0x7f809800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f8087fff640 1 -- 172.21.15.105:0/3950651586 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098003970 con 0x7f80a015f520 2024-03-08T08:29:58.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f8087fff640 1 -- 172.21.15.105:0/3950651586 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8098018070 con 0x7f80a015f520 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f8087fff640 1 -- 172.21.15.105:0/3950651586 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098003d50 con 0x7f80a015f520 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a01f7250 con 0x7f80a015f520 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.694+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a01f7810 con 0x7f80a015f520 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.695+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a006b110 con 0x7f80a015f520 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.695+0000 7f8087fff640 1 -- 172.21.15.105:0/3950651586 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80980050d0 con 0x7f80a015f520 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.696+0000 7f8087fff640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f808c06b4d0 0x7f808c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.696+0000 7f8087fff640 1 -- 172.21.15.105:0/3950651586 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8098089bf0 con 0x7f80a015f520 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.696+0000 7f809f7fe640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f808c06b4d0 0x7f808c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:29:58.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.696+0000 7f809f7fe640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f808c06b4d0 0x7f808c06d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8090009bc0 tx=0x7f8090008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:29:58.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.700+0000 7f8087fff640 1 -- 172.21.15.105:0/3950651586 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80a006b110 con 0x7f80a015f520 2024-03-08T08:29:58.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:58.933+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]} v 0) v1 -- 0x7f80a0145030 con 0x7f80a015f520 2024-03-08T08:29:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.253+0000 7f8087fff640 1 -- 172.21.15.105:0/3950651586 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters"]}]=0 v128) v1 ==== 164+0+0 (secure 0 0 0) 0x7f80a0145030 con 0x7f80a015f520 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f808c06b4d0 msgr2=0x7f808c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f808c06b4d0 0x7f808c06d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8090009bc0 tx=0x7f8090008040 comp rx=0 tx=0).stop 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80a015f520 msgr2=0x7f80a015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80a015f520 0x7f80a015f9a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f80a00b6a90 tx=0x7f809800ec10 comp rx=0 tx=0).stop 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 shutdown_connections 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f808c06b4d0 0x7f808c06d990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80a015f520 0x7f80a015f9a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a015d3c0 0x7f80a015d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:59.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 --2- 172.21.15.105:0/3950651586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8088003f60 0x7f80a00af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:29:59.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 >> 172.21.15.105:0/3950651586 conn(0x7f80a00b2670 msgr2=0x7f80a00b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:29:59.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 shutdown_connections 2024-03-08T08:29:59.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:29:59.255+0000 7f80a5c47640 1 -- 172.21.15.105:0/3950651586 wait complete. 2024-03-08T08:29:59.268 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients ... ERROR 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner: 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner:ERROR: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_f6f80007dcf9e039bb253dd50a92ba3c71c6943a/qa/tasks/cephfs/test_admin.py", line 70, in test_per_client_labeled_perf_counters 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner: while proceed(): 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ 2024-03-08T08:29:59.269 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: 'wait for counters - client.4853' reached maximum tries (31) after waiting for 30 seconds 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner: 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:Ran 1 test in 110.822s 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner: 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner: 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:ERROR: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:That the per-client labelled perf counters depict the clients 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_joscollin_ceph_f6f80007dcf9e039bb253dd50a92ba3c71c6943a/qa/tasks/cephfs/test_admin.py", line 70, in test_per_client_labeled_perf_counters 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner: while proceed(): 2024-03-08T08:29:59.270 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ 2024-03-08T08:29:59.271 INFO:tasks.cephfs_test_runner: raise MaxWhileTries(error_msg) 2024-03-08T08:29:59.271 INFO:tasks.cephfs_test_runner:teuthology.exceptions.MaxWhileTries: 'wait for counters - client.4853' reached maximum tries (31) after waiting for 30 seconds 2024-03-08T08:29:59.271 INFO:tasks.cephfs_test_runner: 2024-03-08T08:29:59.271 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_joscollin_ceph_f6f80007dcf9e039bb253dd50a92ba3c71c6943a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T08:29:59.467 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=cd312c27103a4b5eaf38a693cff508aa Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_joscollin_ceph_f6f80007dcf9e039bb253dd50a92ba3c71c6943a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-03-08T08:29:59.469 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-08T08:29:59.477 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-08T08:29:59.486 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-08T08:29:59.486 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:29:59.486 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:59.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16795 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T08:29:59.506 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-08T08:29:59.506 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:59.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16795 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-08T08:29:59.561 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:59.561 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:59.580 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1164 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T08:29:59.580 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-08T08:29:59.580 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T08:29:59.640 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056360,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1164 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-08T08:29:59.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:59.640 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-08T08:29:59.655 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-08T08:29:59.656 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T08:29:59.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:59.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-08T08:29:59.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:59.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T08:29:59.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:59.792 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-08T08:29:59.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:59.895 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-08T08:29:59.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:59.910 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-08T08:29:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-03-08T08:29:59.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:29:59.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:29:59.982 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-03-08T08:30:00.000 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-08T08:30:00.000 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-08T08:30:00.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:30:00.000 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-08T08:30:00.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:30:00.072 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-08T08:30:00.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:30:00.144 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-03-08T08:30:00.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:30:00.254 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-03-08T08:30:00.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T08:30:00.269 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-03-08T08:30:00.338 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2024-03-08T08:30:00.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:30:00.340 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-08T08:30:00.349 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-08T08:30:00.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:30:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.456+0000 7f7a42ff4640 1 Processor -- start 2024-03-08T08:30:00.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.456+0000 7f7a42ff4640 1 -- start start 2024-03-08T08:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.457+0000 7f7a42ff4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a3c072440 0x7f7a3c071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.457+0000 7f7a42ff4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7a3c0b4d00 0x7f7a3c15af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.457+0000 7f7a42ff4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c15b940 con 0x7f7a3c15b4a0 2024-03-08T08:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.457+0000 7f7a42ff4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c15bc20 con 0x7f7a3c072440 2024-03-08T08:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.457+0000 7f7a42ff4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c070840 con 0x7f7a3c0b4d00 2024-03-08T08:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.458+0000 7f7a40d90640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a3c072440 0x7f7a3c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.458+0000 7f7a40d90640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7a3c072440 0x7f7a3c071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44666/0 (socket says 172.21.15.105:44666) 2024-03-08T08:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.458+0000 7f7a40d90640 1 -- 172.21.15.105:0/3851539479 learned_addr learned my addr 172.21.15.105:0/3851539479 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.458+0000 7f7a3bfff640 1 --2- 172.21.15.105:0/3851539479 >> v2:172.21.15.144:3300/0 conn(0x7f7a3c0b4d00 0x7f7a3c15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.458+0000 7f7a40d90640 1 -- 172.21.15.105:0/3851539479 >> v2:172.21.15.144:3300/0 conn(0x7f7a3c0b4d00 msgr2=0x7f7a3c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.458+0000 7f7a40d90640 1 --2- 172.21.15.105:0/3851539479 >> v2:172.21.15.144:3300/0 conn(0x7f7a3c0b4d00 0x7f7a3c15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.458+0000 7f7a40d90640 1 -- 172.21.15.105:0/3851539479 >> v1:172.21.15.144:6789/0 conn(0x7f7a3c15b4a0 legacy=0x7f7a3c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.458+0000 7f7a40d90640 1 -- 172.21.15.105:0/3851539479 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c070b20 con 0x7f7a3c072440 2024-03-08T08:30:00.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a40d90640 1 --2- 172.21.15.105:0/3851539479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c071020 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0cf2b0 tx=0x7f7a3002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66b3a7239ea96f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:00.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a3b7fe640 1 -- 172.21.15.105:0/3851539479 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3003b070 con 0x7f7a3c072440 2024-03-08T08:30:00.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a3b7fe640 1 -- 172.21.15.105:0/3851539479 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a30036440 con 0x7f7a3c072440 2024-03-08T08:30:00.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a3b7fe640 1 -- 172.21.15.105:0/3851539479 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30036810 con 0x7f7a3c072440 2024-03-08T08:30:00.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a42ff4640 1 -- 172.21.15.105:0/3851539479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 msgr2=0x7f7a3c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/3851539479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c071020 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0cf2b0 tx=0x7f7a3002e170 comp rx=0 tx=0).stop 2024-03-08T08:30:00.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a42ff4640 1 -- 172.21.15.105:0/3851539479 shutdown_connections 2024-03-08T08:30:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/3851539479 >> v2:172.21.15.144:3300/0 conn(0x7f7a3c0b4d00 0x7f7a3c15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/3851539479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c071020 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.459+0000 7f7a42ff4640 1 -- 172.21.15.105:0/3851539479 >> 172.21.15.105:0/3851539479 conn(0x7f7a3c0b1740 msgr2=0x7f7a3c0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.460+0000 7f7a42ff4640 1 -- 172.21.15.105:0/3851539479 shutdown_connections 2024-03-08T08:30:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.460+0000 7f7a42ff4640 1 -- 172.21.15.105:0/3851539479 wait complete. 2024-03-08T08:30:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.461+0000 7f7a42ff4640 1 Processor -- start 2024-03-08T08:30:00.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.461+0000 7f7a42ff4640 1 -- start start 2024-03-08T08:30:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a42ff4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a42ff4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a3c0b4d00 0x7f7a3c154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a42ff4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a3c15b4a0 0x7f7a3c154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a42ff4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0729a0 con 0x7f7a3c072440 2024-03-08T08:30:00.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a42ff4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c072820 con 0x7f7a3c0b4d00 2024-03-08T08:30:00.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a42ff4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c06e7b0 con 0x7f7a3c15b4a0 2024-03-08T08:30:00.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c1f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44678/0 (socket says 172.21.15.105:44678) 2024-03-08T08:30:00.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 -- 172.21.15.105:0/4052302197 learned_addr learned my addr 172.21.15.105:0/4052302197 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:00.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a3bfff640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a3c0b4d00 0x7f7a3c154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a41591640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a3c15b4a0 0x7f7a3c154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 -- 172.21.15.105:0/4052302197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a3c15b4a0 msgr2=0x7f7a3c154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a3c15b4a0 0x7f7a3c154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 -- 172.21.15.105:0/4052302197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a3c0b4d00 msgr2=0x7f7a3c154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a3c0b4d00 0x7f7a3c154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 -- 172.21.15.105:0/4052302197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c154c50 con 0x7f7a3c072440 2024-03-08T08:30:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a40d90640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c1f3a00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7a3002e1b0 tx=0x7f7a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:00.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a397fa640 1 -- 172.21.15.105:0/4052302197 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3003b070 con 0x7f7a3c072440 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a397fa640 1 -- 172.21.15.105:0/4052302197 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3003d050 con 0x7f7a3c072440 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c1442f0 con 0x7f7a3c072440 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a397fa640 1 -- 172.21.15.105:0/4052302197 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30034a90 con 0x7f7a3c072440 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.462+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c147050 con 0x7f7a3c072440 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.463+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c075120 con 0x7f7a3c072440 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.464+0000 7f7a397fa640 1 -- 172.21.15.105:0/4052302197 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a30048060 con 0x7f7a3c072440 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.464+0000 7f7a397fa640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a2006b500 0x7f7a2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.464+0000 7f7a397fa640 1 -- 172.21.15.105:0/4052302197 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7a300ab970 con 0x7f7a3c072440 2024-03-08T08:30:00.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.464+0000 7f7a3bfff640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a2006b500 0x7f7a2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.465+0000 7f7a3bfff640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a2006b500 0x7f7a2006d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0cb9f0 tx=0x7f7a28009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:00.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.469+0000 7f7a397fa640 1 -- 172.21.15.105:0/4052302197 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a3007a4a0 con 0x7f7a3c072440 2024-03-08T08:30:00.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.639+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a3c15ac30 con 0x7f7a2006b500 2024-03-08T08:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.640+0000 7f7a397fa640 1 -- 172.21.15.105:0/4052302197 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13433 (secure 0 0 0) 0x7f7a3c15ac30 con 0x7f7a2006b500 2024-03-08T08:30:00.641 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:30:00.641 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:30:00.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a2006b500 msgr2=0x7f7a2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a2006b500 0x7f7a2006d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0cb9f0 tx=0x7f7a28009190 comp rx=0 tx=0).stop 2024-03-08T08:30:00.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 msgr2=0x7f7a3c1f3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c1f3a00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7a3002e1b0 tx=0x7f7a300046c0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 shutdown_connections 2024-03-08T08:30:00.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f7a2006b500 0x7f7a2006d9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a3c15b4a0 0x7f7a3c154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a3c0b4d00 0x7f7a3c154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 --2- 172.21.15.105:0/4052302197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a3c072440 0x7f7a3c1f3a00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 >> 172.21.15.105:0/4052302197 conn(0x7f7a3c0b1740 msgr2=0x7f7a3c06e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:00.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 shutdown_connections 2024-03-08T08:30:00.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.642+0000 7f7a42ff4640 1 -- 172.21.15.105:0/4052302197 wait complete. 2024-03-08T08:30:00.653 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-03-08T08:29:59.127260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270696,"kb_used_data":25440,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754704024,"statfs":{"total":773094113280,"available":772816920576,"internally_reserved":0,"allocated":26050560,"data_stored":15977766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":251055666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6805861,"num_objects":-1023,"num_object_clones":0,"num_object_copies":-2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2127,"num_write_kb":-8651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001283"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T08:29:55.193814+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:29:55.193814+0000","last_peered":"2024-03-08T08:29:55.193814+0000","last_clean":"2024-03-08T08:29:55.193814+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:29:55.193814+0000","last_undegraded":"2024-03-08T08:29:55.193814+0000","last_fullsized":"2024-03-08T08:29:55.193814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2023424,"data_stored":743877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1064960,"data_stored":147703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":1409024,"data_stored":148004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35288,"kb_used_data":4108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336552,"statfs":{"total":96636764160,"available":96600629248,"internally_reserved":0,"allocated":4206592,"data_stored":2760991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43676,"kb_used_data":8340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328164,"statfs":{"total":96636764160,"available":96592039936,"internally_reserved":0,"allocated":8540160,"data_stored":6949897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1232896,"data_stored":147278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41072,"kb_used_data":5736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330768,"statfs":{"total":96636764160,"available":96594706432,"internally_reserved":0,"allocated":5873664,"data_stored":4342522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1699840,"data_stored":737494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:30:00.654 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:30:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.754+0000 7f95d7b6c640 1 Processor -- start 2024-03-08T08:30:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.754+0000 7f95d7b6c640 1 -- start start 2024-03-08T08:30:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d7b6c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d01453d0 0x7f95d00cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d7b6c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95d00cf340 0x7f95d00ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d7b6c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d01457b0 con 0x7f95d00cd050 2024-03-08T08:30:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d7b6c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00b44e0 con 0x7f95d00cf340 2024-03-08T08:30:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d7b6c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00b47c0 con 0x7f95d01453d0 2024-03-08T08:30:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d6109640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f95d00cd050 0x7f95d015f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51856/0 (socket says 172.21.15.105:51856) 2024-03-08T08:30:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d6109640 1 -- 172.21.15.105:0/921017968 learned_addr learned my addr 172.21.15.105:0/921017968 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d5107640 1 --2- 172.21.15.105:0/921017968 >> v2:172.21.15.144:3300/0 conn(0x7f95d00cf340 0x7f95d00ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d5908640 1 --2- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d01453d0 0x7f95d00cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d4906640 1 -- 172.21.15.105:0/921017968 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277301246 0 0) 0x7f95d01457b0 con 0x7f95d00cd050 2024-03-08T08:30:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d4906640 1 -- 172.21.15.105:0/921017968 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95bc003520 con 0x7f95d00cd050 2024-03-08T08:30:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d5107640 1 -- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d01453d0 msgr2=0x7f95d00cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d5107640 1 --2- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d01453d0 0x7f95d00cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d5107640 1 -- 172.21.15.105:0/921017968 >> v1:172.21.15.105:6789/0 conn(0x7f95d00cd050 legacy=0x7f95d015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d5107640 1 -- 172.21.15.105:0/921017968 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00b4aa0 con 0x7f95d00cf340 2024-03-08T08:30:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.755+0000 7f95d5908640 1 --2- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d01453d0 0x7f95d00cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:30:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.756+0000 7f95d5107640 1 --2- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 0x7f95d00ccb10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95c00097a0 tx=0x7f95c002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d77a9c0e07586ba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.756+0000 7f95d4906640 1 -- 172.21.15.105:0/921017968 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0035020 con 0x7f95d00cf340 2024-03-08T08:30:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.756+0000 7f95d4906640 1 -- 172.21.15.105:0/921017968 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c0031830 con 0x7f95d00cf340 2024-03-08T08:30:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.756+0000 7f95d4906640 1 -- 172.21.15.105:0/921017968 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0031be0 con 0x7f95d00cf340 2024-03-08T08:30:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.757+0000 7f95d7b6c640 1 -- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 msgr2=0x7f95d00ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.757+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 0x7f95d00ccb10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95c00097a0 tx=0x7f95c002b8d0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.757+0000 7f95d7b6c640 1 -- 172.21.15.105:0/921017968 shutdown_connections 2024-03-08T08:30:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.757+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 0x7f95d00ccb10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.757+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/921017968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d01453d0 0x7f95d00cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.757+0000 7f95d7b6c640 1 -- 172.21.15.105:0/921017968 >> 172.21.15.105:0/921017968 conn(0x7f95d00b1740 msgr2=0x7f95d00b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.757+0000 7f95d7b6c640 1 -- 172.21.15.105:0/921017968 shutdown_connections 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.757+0000 7f95d7b6c640 1 -- 172.21.15.105:0/921017968 wait complete. 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.758+0000 7f95d7b6c640 1 Processor -- start 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.758+0000 7f95d7b6c640 1 -- start start 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d7b6c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00cd050 0x7f95d015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d7b6c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 0x7f95d00ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d7b6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d01453d0 0x7f95d00addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d7b6c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d006dad0 con 0x7f95d01453d0 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d7b6c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00b4190 con 0x7f95d00cf340 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d7b6c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d006dc50 con 0x7f95d00cd050 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d01453d0 0x7f95d00addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d01453d0 0x7f95d00addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44682/0 (socket says 172.21.15.105:44682) 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 -- 172.21.15.105:0/266234234 learned_addr learned my addr 172.21.15.105:0/266234234 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d5908640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00cd050 0x7f95d015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d5107640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 0x7f95d00ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 -- 172.21.15.105:0/266234234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00cd050 msgr2=0x7f95d015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00cd050 0x7f95d015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 -- 172.21.15.105:0/266234234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 msgr2=0x7f95d00ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 0x7f95d00ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 -- 172.21.15.105:0/266234234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0142e00 con 0x7f95d01453d0 2024-03-08T08:30:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.759+0000 7f95d6109640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d01453d0 0x7f95d00addb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f95c800cb10 tx=0x7f95c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.760+0000 7f95c67fc640 1 -- 172.21.15.105:0/266234234 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8017070 con 0x7f95d01453d0 2024-03-08T08:30:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.760+0000 7f95c67fc640 1 -- 172.21.15.105:0/266234234 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800ac70 con 0x7f95d01453d0 2024-03-08T08:30:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.760+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d0070740 con 0x7f95d01453d0 2024-03-08T08:30:00.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.760+0000 7f95c67fc640 1 -- 172.21.15.105:0/266234234 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800ae60 con 0x7f95d01453d0 2024-03-08T08:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.760+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d0070d00 con 0x7f95d01453d0 2024-03-08T08:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.761+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d0075120 con 0x7f95d01453d0 2024-03-08T08:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.761+0000 7f95c67fc640 1 -- 172.21.15.105:0/266234234 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95c801a070 con 0x7f95d01453d0 2024-03-08T08:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.761+0000 7f95c67fc640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f95b406b500 0x7f95b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.761+0000 7f95c67fc640 1 -- 172.21.15.105:0/266234234 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f95c8089ba0 con 0x7f95d01453d0 2024-03-08T08:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.761+0000 7f95d5908640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f95b406b500 0x7f95b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.762+0000 7f95d5908640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f95b406b500 0x7f95b406d9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009560 tx=0x7f95cc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.766+0000 7f95c67fc640 1 -- 172.21.15.105:0/266234234 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95c8058760 con 0x7f95d01453d0 2024-03-08T08:30:00.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.932+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95d00ccbb0 con 0x7f95b406b500 2024-03-08T08:30:00.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.933+0000 7f95c67fc640 1 -- 172.21.15.105:0/266234234 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13433 (secure 0 0 0) 0x7f95d00ccbb0 con 0x7f95b406b500 2024-03-08T08:30:00.934 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:30:00.934 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:30:00.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f95b406b500 msgr2=0x7f95b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f95b406b500 0x7f95b406d9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009560 tx=0x7f95cc009190 comp rx=0 tx=0).stop 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d01453d0 msgr2=0x7f95d00addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d01453d0 0x7f95d00addb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f95c800cb10 tx=0x7f95c800a060 comp rx=0 tx=0).stop 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 shutdown_connections 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f95b406b500 0x7f95b406d9c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d01453d0 0x7f95d00addb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d00cf340 0x7f95d00ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 --2- 172.21.15.105:0/266234234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95d00cd050 0x7f95d015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:00.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 >> 172.21.15.105:0/266234234 conn(0x7f95d00b1740 msgr2=0x7f95d00b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 shutdown_connections 2024-03-08T08:30:00.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:00.935+0000 7f95d7b6c640 1 -- 172.21.15.105:0/266234234 wait complete. 2024-03-08T08:30:00.947 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-03-08T08:29:59.127260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270696,"kb_used_data":25440,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754704024,"statfs":{"total":773094113280,"available":772816920576,"internally_reserved":0,"allocated":26050560,"data_stored":15977766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":251055666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6805861,"num_objects":-1023,"num_object_clones":0,"num_object_copies":-2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2127,"num_write_kb":-8651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001283"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T08:29:55.193814+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:29:55.193814+0000","last_peered":"2024-03-08T08:29:55.193814+0000","last_clean":"2024-03-08T08:29:55.193814+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:29:55.193814+0000","last_undegraded":"2024-03-08T08:29:55.193814+0000","last_fullsized":"2024-03-08T08:29:55.193814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2023424,"data_stored":743877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1064960,"data_stored":147703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":1409024,"data_stored":148004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35288,"kb_used_data":4108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336552,"statfs":{"total":96636764160,"available":96600629248,"internally_reserved":0,"allocated":4206592,"data_stored":2760991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43676,"kb_used_data":8340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328164,"statfs":{"total":96636764160,"available":96592039936,"internally_reserved":0,"allocated":8540160,"data_stored":6949897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1232896,"data_stored":147278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41072,"kb_used_data":5736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330768,"statfs":{"total":96636764160,"available":96594706432,"internally_reserved":0,"allocated":5873664,"data_stored":4342522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1699840,"data_stored":737494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:30:00.947 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-08T08:30:00.947 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:30:01.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.046+0000 7fe822580640 1 Processor -- start 2024-03-08T08:30:01.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.047+0000 7fe822580640 1 -- start start 2024-03-08T08:30:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.047+0000 7fe822580640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c161850 0x7fe81c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.047+0000 7fe822580640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c0d48a0 con 0x7fe81c06bcb0 2024-03-08T08:30:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.047+0000 7fe822580640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c0d4b80 con 0x7fe81c161470 2024-03-08T08:30:01.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.047+0000 7fe822580640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06c170 con 0x7fe81c161850 2024-03-08T08:30:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.047+0000 7fe81b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe81c06bcb0 0x7fe81c0b2450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51864/0 (socket says 172.21.15.105:51864) 2024-03-08T08:30:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.047+0000 7fe81b7fe640 1 -- 172.21.15.105:0/2687256541 learned_addr learned my addr 172.21.15.105:0/2687256541 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969587789 0 0) 0x7fe81c0d48a0 con 0x7fe81c06bcb0 2024-03-08T08:30:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe820b1d640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c161850 0x7fe81c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe804003520 con 0x7fe81c06bcb0 2024-03-08T08:30:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3063958722 0 0) 0x7fe804003520 con 0x7fe81c06bcb0 2024-03-08T08:30:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c161850 msgr2=0x7fe81c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c161850 0x7fe81c06c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 >> v1:172.21.15.144:6789/0 conn(0x7fe81c161470 legacy=0x7fe81c06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c06c450 con 0x7fe81c06bcb0 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fe810001670 con 0x7fe81c06bcb0 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 >> v1:172.21.15.105:6789/0 conn(0x7fe81c06bcb0 legacy=0x7fe81c0b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c161850 0x7fe8040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804003f20 0x7fe804004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe804004d40 0x7fe8040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06c170 con 0x7fe804004d40 2024-03-08T08:30:01.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80400bad0 con 0x7fe804003f20 2024-03-08T08:30:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80400bcd0 con 0x7fe81c161850 2024-03-08T08:30:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe820b1d640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe804004d40 0x7fe8040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81b7fe640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804003f20 0x7fe804004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe81bfff640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c161850 0x7fe8040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe820b1d640 1 -- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c161850 msgr2=0x7fe8040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe820b1d640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c161850 0x7fe8040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe820b1d640 1 -- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804003f20 msgr2=0x7fe804004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe820b1d640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804003f20 0x7fe804004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.048+0000 7fe820b1d640 1 -- 172.21.15.105:0/2687256541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe804003520 con 0x7fe804004d40 2024-03-08T08:30:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.049+0000 7fe820b1d640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe804004d40 0x7fe8040051c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe814009fc0 tx=0x7fe804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70afce9f098ed2cb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.049+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814039070 con 0x7fe804004d40 2024-03-08T08:30:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.049+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81402cc20 con 0x7fe804004d40 2024-03-08T08:30:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.049+0000 7fe81affd640 1 -- 172.21.15.105:0/2687256541 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814009c10 con 0x7fe804004d40 2024-03-08T08:30:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.050+0000 7fe822580640 1 -- 172.21.15.105:0/2687256541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe804004d40 msgr2=0x7fe8040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.050+0000 7fe822580640 1 --2- 172.21.15.105:0/2687256541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe804004d40 0x7fe8040051c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe814009fc0 tx=0x7fe804005830 comp rx=0 tx=0).stop 2024-03-08T08:30:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.050+0000 7fe820b1d640 1 -- 172.21.15.105:0/2687256541 reap_dead start 2024-03-08T08:30:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.050+0000 7fe822580640 1 -- 172.21.15.105:0/2687256541 shutdown_connections 2024-03-08T08:30:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.050+0000 7fe822580640 1 -- 172.21.15.105:0/2687256541 >> 172.21.15.105:0/2687256541 conn(0x7fe81c0d3ce0 msgr2=0x7fe81c0d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.050+0000 7fe822580640 1 -- 172.21.15.105:0/2687256541 shutdown_connections 2024-03-08T08:30:01.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.050+0000 7fe822580640 1 -- 172.21.15.105:0/2687256541 wait complete. 2024-03-08T08:30:01.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.051+0000 7fe822580640 1 Processor -- start 2024-03-08T08:30:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.051+0000 7fe822580640 1 -- start start 2024-03-08T08:30:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe822580640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c1541d0 0x7fe81c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe822580640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe81c1f3920 0x7fe81c15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe822580640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81c0a0ac0 0x7fe81c0a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe822580640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06d4f0 con 0x7fe81c1f3920 2024-03-08T08:30:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe822580640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06d370 con 0x7fe81c0a0ac0 2024-03-08T08:30:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe822580640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06d670 con 0x7fe81c1541d0 2024-03-08T08:30:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe81c1f3920 0x7fe81c15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe81c1f3920 0x7fe81c15ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44700/0 (socket says 172.21.15.105:44700) 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 -- 172.21.15.105:0/2611768657 learned_addr learned my addr 172.21.15.105:0/2611768657 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81bfff640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c1541d0 0x7fe81c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe820b1d640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81c0a0ac0 0x7fe81c0a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 -- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c1541d0 msgr2=0x7fe81c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c1541d0 0x7fe81c15fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 -- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81c0a0ac0 msgr2=0x7fe81c0a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81c0a0ac0 0x7fe81c0a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 -- 172.21.15.105:0/2611768657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0a1880 con 0x7fe81c1f3920 2024-03-08T08:30:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe820b1d640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81c0a0ac0 0x7fe81c0a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:30:01.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81bfff640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c1541d0 0x7fe81c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:30:01.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe81b7fe640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe81c1f3920 0x7fe81c15ffb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe810007290 tx=0x7fe810007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:01.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.052+0000 7fe818ff9640 1 -- 172.21.15.105:0/2611768657 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000f9c0 con 0x7fe81c1f3920 2024-03-08T08:30:01.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.053+0000 7fe818ff9640 1 -- 172.21.15.105:0/2611768657 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810018070 con 0x7fe81c1f3920 2024-03-08T08:30:01.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.053+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c0c8be0 con 0x7fe81c1f3920 2024-03-08T08:30:01.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.053+0000 7fe818ff9640 1 -- 172.21.15.105:0/2611768657 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000fda0 con 0x7fe81c1f3920 2024-03-08T08:30:01.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.053+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c0c91a0 con 0x7fe81c1f3920 2024-03-08T08:30:01.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.054+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c075120 con 0x7fe81c1f3920 2024-03-08T08:30:01.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.054+0000 7fe818ff9640 1 -- 172.21.15.105:0/2611768657 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe81001d460 con 0x7fe81c1f3920 2024-03-08T08:30:01.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.054+0000 7fe818ff9640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe80806b500 0x7fe80806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.054+0000 7fe818ff9640 1 -- 172.21.15.105:0/2611768657 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe810089f20 con 0x7fe81c1f3920 2024-03-08T08:30:01.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.054+0000 7fe81bfff640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe80806b500 0x7fe80806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.055+0000 7fe81bfff640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe80806b500 0x7fe80806d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0037e0 tx=0x7fe80c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:01.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.059+0000 7fe818ff9640 1 -- 172.21.15.105:0/2611768657 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8100590a0 con 0x7fe81c1f3920 2024-03-08T08:30:01.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.226+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe81c0d43a0 con 0x7fe80806b500 2024-03-08T08:30:01.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.227+0000 7fe818ff9640 1 -- 172.21.15.105:0/2611768657 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13449 (secure 0 0 0) 0x7fe81c0d43a0 con 0x7fe80806b500 2024-03-08T08:30:01.228 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:30:01.229 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:30:01.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe80806b500 msgr2=0x7fe80806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe80806b500 0x7fe80806d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0037e0 tx=0x7fe80c009040 comp rx=0 tx=0).stop 2024-03-08T08:30:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe81c1f3920 msgr2=0x7fe81c15ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe81c1f3920 0x7fe81c15ffb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe810007290 tx=0x7fe810007750 comp rx=0 tx=0).stop 2024-03-08T08:30:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 shutdown_connections 2024-03-08T08:30:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fe80806b500 0x7fe80806d9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe81c0a0ac0 0x7fe81c0a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe81c1f3920 0x7fe81c15ffb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 --2- 172.21.15.105:0/2611768657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe81c1541d0 0x7fe81c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 >> 172.21.15.105:0/2611768657 conn(0x7fe81c0d3ce0 msgr2=0x7fe81c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 shutdown_connections 2024-03-08T08:30:01.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.229+0000 7fe822580640 1 -- 172.21.15.105:0/2611768657 wait complete. 2024-03-08T08:30:01.241 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-03-08T08:30:01.127514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":257448,"kb_used_data":12180,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754717272,"statfs":{"total":773094113280,"available":772830486528,"internally_reserved":0,"allocated":12472320,"data_stored":5013861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":251055666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6805861,"num_objects":-1023,"num_object_clones":0,"num_object_copies":-2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2127,"num_write_kb":-8651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001093"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-08T08:29:55.193814+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:29:55.193814+0000","last_peered":"2024-03-08T08:29:55.193814+0000","last_clean":"2024-03-08T08:29:55.193814+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:29:55.193814+0000","last_undegraded":"2024-03-08T08:29:55.193814+0000","last_fullsized":"2024-03-08T08:29:55.193814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2023424,"data_stored":743877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1064960,"data_stored":147703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":1409024,"data_stored":148004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35288,"kb_used_data":4108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336552,"statfs":{"total":96636764160,"available":96600629248,"internally_reserved":0,"allocated":4206592,"data_stored":2760991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35752,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336088,"statfs":{"total":96636764160,"available":96600154112,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1232896,"data_stored":147278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35748,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336092,"statfs":{"total":96636764160,"available":96600158208,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1699840,"data_stored":737494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:30:01.242 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T08:30:01.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.342+0000 7f5b2aa6b640 1 Processor -- start 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.342+0000 7f5b2aa6b640 1 -- start start 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.343+0000 7f5b2aa6b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b24161470 0x7f5b2406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.343+0000 7f5b2aa6b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b2406bcb0 0x7f5b240d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.343+0000 7f5b2aa6b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b240d48a0 con 0x7f5b24161850 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.343+0000 7f5b2aa6b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b240d2ba0 con 0x7f5b24161470 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.343+0000 7f5b2aa6b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240d4b80 con 0x7f5b2406bcb0 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.343+0000 7f5b28807640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b24161470 0x7f5b2406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.343+0000 7f5b28807640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b24161470 0x7f5b2406c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44702/0 (socket says 172.21.15.105:44702) 2024-03-08T08:30:01.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.343+0000 7f5b28807640 1 -- 172.21.15.105:0/3542595221 learned_addr learned my addr 172.21.15.105:0/3542595221 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.344+0000 7f5b29008640 1 --2- 172.21.15.105:0/3542595221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b2406bcb0 0x7f5b240d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.344+0000 7f5b28807640 1 -- 172.21.15.105:0/3542595221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b2406bcb0 msgr2=0x7f5b240d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.344+0000 7f5b28807640 1 --2- 172.21.15.105:0/3542595221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b2406bcb0 0x7f5b240d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.344+0000 7f5b28807640 1 -- 172.21.15.105:0/3542595221 >> v1:172.21.15.144:6789/0 conn(0x7f5b24161850 legacy=0x7f5b2406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.344+0000 7f5b28807640 1 -- 172.21.15.105:0/3542595221 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2406c170 con 0x7f5b24161470 2024-03-08T08:30:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.344+0000 7f5b28807640 1 --2- 172.21.15.105:0/3542595221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b24161470 0x7f5b2406c830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009870 tx=0x7f5b1402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dabc232780760c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b237fe640 1 -- 172.21.15.105:0/3542595221 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14035020 con 0x7f5b24161470 2024-03-08T08:30:01.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b237fe640 1 -- 172.21.15.105:0/3542595221 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b14031640 con 0x7f5b24161470 2024-03-08T08:30:01.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b237fe640 1 -- 172.21.15.105:0/3542595221 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1402d040 con 0x7f5b24161470 2024-03-08T08:30:01.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/3542595221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b24161470 msgr2=0x7f5b2406c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/3542595221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b24161470 0x7f5b2406c830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009870 tx=0x7f5b1402b8f0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/3542595221 shutdown_connections 2024-03-08T08:30:01.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/3542595221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b2406bcb0 0x7f5b240d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/3542595221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b24161470 0x7f5b2406c830 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/3542595221 >> 172.21.15.105:0/3542595221 conn(0x7f5b240d3ce0 msgr2=0x7f5b240d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:01.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/3542595221 shutdown_connections 2024-03-08T08:30:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.345+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/3542595221 wait complete. 2024-03-08T08:30:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.346+0000 7f5b2aa6b640 1 Processor -- start 2024-03-08T08:30:01.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.346+0000 7f5b2aa6b640 1 -- start start 2024-03-08T08:30:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b2aa6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2406bcb0 0x7f5b2409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b2aa6b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b24161470 0x7f5b240c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b2aa6b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b24161850 0x7f5b240ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b2aa6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2406cf40 con 0x7f5b2406bcb0 2024-03-08T08:30:01.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b2aa6b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2406d370 con 0x7f5b24161470 2024-03-08T08:30:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b2aa6b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241f4b30 con 0x7f5b24161850 2024-03-08T08:30:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b28807640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2406bcb0 0x7f5b2409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b28807640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2406bcb0 0x7f5b2409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44704/0 (socket says 172.21.15.105:44704) 2024-03-08T08:30:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.347+0000 7f5b28807640 1 -- 172.21.15.105:0/2321542974 learned_addr learned my addr 172.21.15.105:0/2321542974 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b29008640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b24161850 0x7f5b240ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b23fff640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b24161470 0x7f5b240c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b28807640 1 -- 172.21.15.105:0/2321542974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b24161850 msgr2=0x7f5b240ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b28807640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b24161850 0x7f5b240ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b28807640 1 -- 172.21.15.105:0/2321542974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b24161470 msgr2=0x7f5b240c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b28807640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b24161470 0x7f5b240c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b28807640 1 -- 172.21.15.105:0/2321542974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2409dd90 con 0x7f5b2406bcb0 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b28807640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2406bcb0 0x7f5b2409fd60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5b14000c00 tx=0x7f5b140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b217fa640 1 -- 172.21.15.105:0/2321542974 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14040070 con 0x7f5b2406bcb0 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b217fa640 1 -- 172.21.15.105:0/2321542974 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b140399f0 con 0x7f5b2406bcb0 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b217fa640 1 -- 172.21.15.105:0/2321542974 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14035040 con 0x7f5b2406bcb0 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2409e070 con 0x7f5b2406bcb0 2024-03-08T08:30:01.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.348+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2409e630 con 0x7f5b2406bcb0 2024-03-08T08:30:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.349+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b24075120 con 0x7f5b2406bcb0 2024-03-08T08:30:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.349+0000 7f5b217fa640 1 -- 172.21.15.105:0/2321542974 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b1404b060 con 0x7f5b2406bcb0 2024-03-08T08:30:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.350+0000 7f5b217fa640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5b1006b4d0 0x7f5b1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.350+0000 7f5b217fa640 1 -- 172.21.15.105:0/2321542974 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5b140ab8a0 con 0x7f5b2406bcb0 2024-03-08T08:30:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.350+0000 7f5b23fff640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5b1006b4d0 0x7f5b1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.350+0000 7f5b23fff640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5b1006b4d0 0x7f5b1006d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b24160170 tx=0x7f5b18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:01.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.354+0000 7f5b217fa640 1 -- 172.21.15.105:0/2321542974 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b1407aa20 con 0x7f5b2406bcb0 2024-03-08T08:30:01.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.525+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b24067ec0 con 0x7f5b2406bcb0 2024-03-08T08:30:01.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.526+0000 7f5b217fa640 1 -- 172.21.15.105:0/2321542974 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13034 (secure 0 0 0) 0x7f5b1407a320 con 0x7f5b2406bcb0 2024-03-08T08:30:01.527 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:30:01.528 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":36,"fsid":"1afff3d7-46c3-4cd2-9212-c89ee069e495","created":"2024-03-08T08:27:14.916064+0000","modified":"2024-03-08T08:29:58.246339+0000","last_up_change":"2024-03-08T08:27:26.035656+0000","last_in_change":"2024-03-08T08:27:17.109283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T08:27:27.135900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32540e82-9419-45ae-840e-488ebfd5fdeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6813","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6815","nonce":491346206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6819","nonce":491346206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6823","nonce":491346206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":491346206},{"type":"v1","addr":"172.21.15.105:6821","nonce":491346206}]},"public_addr":"172.21.15.105:6815/491346206","cluster_addr":"172.21.15.105:6819/491346206","heartbeat_back_addr":"172.21.15.105:6823/491346206","heartbeat_front_addr":"172.21.15.105:6821/491346206","state":["exists","up"]},{"osd":1,"uuid":"2e4be82f-332a-4c95-8a81-b7a186b0c433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6825","nonce":2989492650}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6827","nonce":2989492650}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6831","nonce":2989492650}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2989492650},{"type":"v1","addr":"172.21.15.105:6829","nonce":2989492650}]},"public_addr":"172.21.15.105:6825/2989492650","cluster_addr":"172.21.15.105:6827/2989492650","heartbeat_back_addr":"172.21.15.105:6831/2989492650","heartbeat_front_addr":"172.21.15.105:6829/2989492650","state":["exists","up"]},{"osd":2,"uuid":"7475a6f5-267f-4139-83f9-568188376e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6801","nonce":1086051235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6803","nonce":1086051235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6807","nonce":1086051235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":1086051235},{"type":"v1","addr":"172.21.15.105:6805","nonce":1086051235}]},"public_addr":"172.21.15.105:6801/1086051235","cluster_addr":"172.21.15.105:6803/1086051235","heartbeat_back_addr":"172.21.15.105:6807/1086051235","heartbeat_front_addr":"172.21.15.105:6805/1086051235","state":["exists","up"]},{"osd":3,"uuid":"68fae46e-aba8-42fe-a4db-693879d6aa8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6809","nonce":2646109234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6811","nonce":2646109234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6817","nonce":2646109234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2646109234},{"type":"v1","addr":"172.21.15.105:6814","nonce":2646109234}]},"public_addr":"172.21.15.105:6809/2646109234","cluster_addr":"172.21.15.105:6811/2646109234","heartbeat_back_addr":"172.21.15.105:6817/2646109234","heartbeat_front_addr":"172.21.15.105:6814/2646109234","state":["exists","up"]},{"osd":4,"uuid":"c49af92b-b753-48dd-9ac9-11c95482035f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6801","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6802","nonce":3800656202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6805","nonce":3800656202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6812","nonce":3800656202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6807","nonce":3800656202},{"type":"v1","addr":"172.21.15.144:6809","nonce":3800656202}]},"public_addr":"172.21.15.144:6802/3800656202","cluster_addr":"172.21.15.144:6805/3800656202","heartbeat_back_addr":"172.21.15.144:6812/3800656202","heartbeat_front_addr":"172.21.15.144:6809/3800656202","state":["exists","up"]},{"osd":5,"uuid":"50a656b3-a9ed-4c0a-b80c-a6706559b239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6817","nonce":56982278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6819","nonce":56982278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6823","nonce":56982278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":56982278},{"type":"v1","addr":"172.21.15.144:6821","nonce":56982278}]},"public_addr":"172.21.15.144:6817/56982278","cluster_addr":"172.21.15.144:6819/56982278","heartbeat_back_addr":"172.21.15.144:6823/56982278","heartbeat_front_addr":"172.21.15.144:6821/56982278","state":["exists","up"]},{"osd":6,"uuid":"037ceb75-aa38-4cc5-a3ef-7a2e48d185fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6803","nonce":768956787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6808","nonce":768956787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6815","nonce":768956787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6811","nonce":768956787},{"type":"v1","addr":"172.21.15.144:6813","nonce":768956787}]},"public_addr":"172.21.15.144:6803/768956787","cluster_addr":"172.21.15.144:6808/768956787","heartbeat_back_addr":"172.21.15.144:6815/768956787","heartbeat_front_addr":"172.21.15.144:6813/768956787","state":["exists","up"]},{"osd":7,"uuid":"9feb8651-f7cd-474c-9b63-d51954617f4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6825","nonce":1634585826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6827","nonce":1634585826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6831","nonce":1634585826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1634585826},{"type":"v1","addr":"172.21.15.144:6829","nonce":1634585826}]},"public_addr":"172.21.15.144:6825/1634585826","cluster_addr":"172.21.15.144:6827/1634585826","heartbeat_back_addr":"172.21.15.144:6831/1634585826","heartbeat_front_addr":"172.21.15.144:6829/1634585826","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.296312+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.342548+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.267987+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.269750+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.254392+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226561+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.226559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T08:27:22.274642+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6837/2431299564":"2024-03-09T08:29:55.189119+0000","172.21.15.105:6836/2431299564":"2024-03-09T08:29:55.189119+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T08:30:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5b1006b4d0 msgr2=0x7f5b1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5b1006b4d0 0x7f5b1006d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b24160170 tx=0x7f5b18009040 comp rx=0 tx=0).stop 2024-03-08T08:30:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2406bcb0 msgr2=0x7f5b2409fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2406bcb0 0x7f5b2409fd60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5b14000c00 tx=0x7f5b140046c0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 shutdown_connections 2024-03-08T08:30:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5b1006b4d0 0x7f5b1006d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b24161850 0x7f5b240ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b24161470 0x7f5b240c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 --2- 172.21.15.105:0/2321542974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b2406bcb0 0x7f5b2409fd60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:01.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 >> 172.21.15.105:0/2321542974 conn(0x7f5b240d3ce0 msgr2=0x7f5b240ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 shutdown_connections 2024-03-08T08:30:01.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:01.528+0000 7f5b2aa6b640 1 -- 172.21.15.105:0/2321542974 wait complete. 2024-03-08T08:30:02.541 INFO:tasks.ceph:Scrubbing osd.0 2024-03-08T08:30:02.541 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-08T08:30:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.653+0000 7f720712e640 1 Processor -- start 2024-03-08T08:30:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.653+0000 7f720712e640 1 -- start start 2024-03-08T08:30:02.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f720712e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7200161b10 0x7f72000d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f720712e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720006e960 con 0x7f7200161ef0 2024-03-08T08:30:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f720712e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72000d07a0 con 0x7f720006f3d0 2024-03-08T08:30:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f720712e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720006ebd0 con 0x7f7200161b10 2024-03-08T08:30:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f7204eca640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7200161b10 0x7f72000d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f71fffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7200161ef0 0x7f72001623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51876/0 (socket says 172.21.15.105:51876) 2024-03-08T08:30:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f71fffff640 1 -- 172.21.15.105:0/1119223057 learned_addr learned my addr 172.21.15.105:0/1119223057 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f7204eca640 1 -- 172.21.15.105:0/1119223057 >> v1:172.21.15.144:6789/0 conn(0x7f720006f3d0 legacy=0x7f720006f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:02.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f7204eca640 1 -- 172.21.15.105:0/1119223057 >> v1:172.21.15.105:6789/0 conn(0x7f7200161ef0 legacy=0x7f72001623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f7204eca640 1 -- 172.21.15.105:0/1119223057 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720006eeb0 con 0x7f7200161b10 2024-03-08T08:30:02.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.654+0000 7f71ff7fe640 1 -- 172.21.15.105:0/1119223057 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334966902 0 0) 0x7f720006e960 con 0x7f7200161ef0 2024-03-08T08:30:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.655+0000 7f7204eca640 1 --2- 172.21.15.105:0/1119223057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161b10 0x7f72000d0110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f71f00097d0 tx=0x7f71f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b93b93bf095e98 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.655+0000 7f71ff7fe640 1 -- 172.21.15.105:0/1119223057 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0035020 con 0x7f7200161b10 2024-03-08T08:30:02.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.655+0000 7f71ff7fe640 1 -- 172.21.15.105:0/1119223057 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0031830 con 0x7f7200161b10 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.655+0000 7f71ff7fe640 1 -- 172.21.15.105:0/1119223057 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0031c00 con 0x7f7200161b10 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.656+0000 7f720712e640 1 -- 172.21.15.105:0/1119223057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161b10 msgr2=0x7f72000d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.656+0000 7f720712e640 1 --2- 172.21.15.105:0/1119223057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161b10 0x7f72000d0110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f71f00097d0 tx=0x7f71f002b8d0 comp rx=0 tx=0).stop 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.656+0000 7f720712e640 1 -- 172.21.15.105:0/1119223057 shutdown_connections 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.656+0000 7f720712e640 1 --2- 172.21.15.105:0/1119223057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161b10 0x7f72000d0110 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.656+0000 7f720712e640 1 -- 172.21.15.105:0/1119223057 >> 172.21.15.105:0/1119223057 conn(0x7f72000b36c0 msgr2=0x7f72000b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.656+0000 7f720712e640 1 -- 172.21.15.105:0/1119223057 shutdown_connections 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.656+0000 7f720712e640 1 -- 172.21.15.105:0/1119223057 wait complete. 2024-03-08T08:30:02.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.657+0000 7f720712e640 1 Processor -- start 2024-03-08T08:30:02.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.657+0000 7f720712e640 1 -- start start 2024-03-08T08:30:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f720712e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f720006f3d0 0x7f72001f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f720712e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7200161b10 0x7f72001f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f720712e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161ef0 0x7f72001f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f720712e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72000d0c00 con 0x7f7200161ef0 2024-03-08T08:30:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f720712e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72000d0a80 con 0x7f7200161b10 2024-03-08T08:30:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f720712e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72000ced20 con 0x7f720006f3d0 2024-03-08T08:30:02.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161ef0 0x7f72001f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161ef0 0x7f72001f6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44716/0 (socket says 172.21.15.105:44716) 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 -- 172.21.15.105:0/174771665 learned_addr learned my addr 172.21.15.105:0/174771665 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f71fffff640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7200161b10 0x7f72001f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f7204eca640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f720006f3d0 0x7f72001f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 -- 172.21.15.105:0/174771665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f720006f3d0 msgr2=0x7f72001f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f720006f3d0 0x7f72001f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 -- 172.21.15.105:0/174771665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7200161b10 msgr2=0x7f72001f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7200161b10 0x7f72001f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:02.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 -- 172.21.15.105:0/174771665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72001f4900 con 0x7f7200161ef0 2024-03-08T08:30:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.658+0000 7f72056cb640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161ef0 0x7f72001f6730 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c6b0 tx=0x7f71f800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.659+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8003b70 con 0x7f7200161ef0 2024-03-08T08:30:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.659+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8003d30 con 0x7f7200161ef0 2024-03-08T08:30:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.659+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8005310 con 0x7f7200161ef0 2024-03-08T08:30:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.659+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72000c5ac0 con 0x7f7200161ef0 2024-03-08T08:30:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.659+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72000c6080 con 0x7f7200161ef0 2024-03-08T08:30:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.659+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7200075200 con 0x7f7200161ef0 2024-03-08T08:30:02.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.660+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71f80054b0 con 0x7f7200161ef0 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.660+0000 7f71fd7fa640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f71ec06b4d0 0x7f71ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.661+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f71f808a670 con 0x7f7200161ef0 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.661+0000 7f7204eca640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f71ec06b4d0 0x7f71ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.661+0000 7f71fd7fa640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f71ec071390 0x7f71ec0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.661+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 --> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f71f8010e50 con 0x7f71ec071390 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.661+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f808ab00 con 0x7f7200161ef0 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.661+0000 7f71fffff640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f71ec071390 0x7f71ec0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.661+0000 7f7204eca640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f71ec06b4d0 0x7f71ec06d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f71f0036000 tx=0x7f71f003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:02.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.661+0000 7f71fffff640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f71ec071390 0x7f71ec0737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:02.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.662+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 <== osd.0 v2:172.21.15.105:6813/491346206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f71f8010e50 con 0x7f71ec071390 2024-03-08T08:30:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 --> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7200072b00 con 0x7f71ec071390 2024-03-08T08:30:02.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f71fd7fa640 1 -- 172.21.15.105:0/174771665 <== osd.0 v2:172.21.15.105:6813/491346206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7200072b00 con 0x7f71ec071390 2024-03-08T08:30:02.676 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:30:02.676 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T08:30:02.676 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:30:02.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f71ec071390 msgr2=0x7f71ec0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6813/491346206,v1:172.21.15.105:6815/491346206] conn(0x7f71ec071390 0x7f71ec0737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f71ec06b4d0 msgr2=0x7f71ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f71ec06b4d0 0x7f71ec06d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f71f0036000 tx=0x7f71f003d040 comp rx=0 tx=0).stop 2024-03-08T08:30:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161ef0 msgr2=0x7f72001f6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 --2- 172.21.15.105:0/174771665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7200161ef0 0x7f72001f6730 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c6b0 tx=0x7f71f800cb70 comp rx=0 tx=0).stop 2024-03-08T08:30:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f72056cb640 1 -- 172.21.15.105:0/174771665 reap_dead start 2024-03-08T08:30:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 shutdown_connections 2024-03-08T08:30:02.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 >> 172.21.15.105:0/174771665 conn(0x7f72000b36c0 msgr2=0x7f720006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 shutdown_connections 2024-03-08T08:30:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.675+0000 7f720712e640 1 -- 172.21.15.105:0/174771665 wait complete. 2024-03-08T08:30:02.687 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-08T08:30:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.786+0000 7f5be71b6640 1 Processor -- start 2024-03-08T08:30:03.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.786+0000 7f5be71b6640 1 -- start start 2024-03-08T08:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be71b6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5be0072440 0x7f5be0071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be71b6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be00b4d00 0x7f5be015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be71b6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be015b940 con 0x7f5be015b4a0 2024-03-08T08:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be71b6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be015bc20 con 0x7f5be0072440 2024-03-08T08:30:03.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be71b6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0070840 con 0x7f5be00b4d00 2024-03-08T08:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be5753640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5be015b4a0 0x7f5be006e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51892/0 (socket says 172.21.15.105:51892) 2024-03-08T08:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be5753640 1 -- 172.21.15.105:0/1389756469 learned_addr learned my addr 172.21.15.105:0/1389756469 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5bdffff640 1 --2- 172.21.15.105:0/1389756469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be00b4d00 0x7f5be015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be4f52640 1 --2- 172.21.15.105:0/1389756469 >> v2:172.21.15.144:3300/0 conn(0x7f5be0072440 0x7f5be0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5bdf7fe640 1 -- 172.21.15.105:0/1389756469 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195416312 0 0) 0x7f5be015b940 con 0x7f5be015b4a0 2024-03-08T08:30:03.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5bdf7fe640 1 -- 172.21.15.105:0/1389756469 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc8003520 con 0x7f5be015b4a0 2024-03-08T08:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5bdffff640 1 -- 172.21.15.105:0/1389756469 >> v2:172.21.15.144:3300/0 conn(0x7f5be0072440 msgr2=0x7f5be0071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5bdffff640 1 --2- 172.21.15.105:0/1389756469 >> v2:172.21.15.144:3300/0 conn(0x7f5be0072440 0x7f5be0071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5bdffff640 1 -- 172.21.15.105:0/1389756469 >> v1:172.21.15.105:6789/0 conn(0x7f5be015b4a0 legacy=0x7f5be006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5bdffff640 1 -- 172.21.15.105:0/1389756469 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0070b20 con 0x7f5be00b4d00 2024-03-08T08:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.787+0000 7f5be4f52640 1 --2- 172.21.15.105:0/1389756469 >> v2:172.21.15.144:3300/0 conn(0x7f5be0072440 0x7f5be0071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:30:03.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.788+0000 7f5bdffff640 1 --2- 172.21.15.105:0/1389756469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be00b4d00 0x7f5be015af60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40097a0 tx=0x7f5bd402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc333d445f2d35a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.788+0000 7f5bdf7fe640 1 -- 172.21.15.105:0/1389756469 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4035020 con 0x7f5be00b4d00 2024-03-08T08:30:03.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.788+0000 7f5bdf7fe640 1 -- 172.21.15.105:0/1389756469 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd4031830 con 0x7f5be00b4d00 2024-03-08T08:30:03.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.788+0000 7f5bdf7fe640 1 -- 172.21.15.105:0/1389756469 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4031c00 con 0x7f5be00b4d00 2024-03-08T08:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.789+0000 7f5be71b6640 1 -- 172.21.15.105:0/1389756469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be00b4d00 msgr2=0x7f5be015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.789+0000 7f5be71b6640 1 --2- 172.21.15.105:0/1389756469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be00b4d00 0x7f5be015af60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40097a0 tx=0x7f5bd402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.789+0000 7f5be71b6640 1 -- 172.21.15.105:0/1389756469 shutdown_connections 2024-03-08T08:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.789+0000 7f5be71b6640 1 --2- 172.21.15.105:0/1389756469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be00b4d00 0x7f5be015af60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.789+0000 7f5be71b6640 1 --2- 172.21.15.105:0/1389756469 >> v2:172.21.15.144:3300/0 conn(0x7f5be0072440 0x7f5be0071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.789+0000 7f5be71b6640 1 -- 172.21.15.105:0/1389756469 >> 172.21.15.105:0/1389756469 conn(0x7f5be00b1740 msgr2=0x7f5be00b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.789+0000 7f5be71b6640 1 -- 172.21.15.105:0/1389756469 shutdown_connections 2024-03-08T08:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.789+0000 7f5be71b6640 1 -- 172.21.15.105:0/1389756469 wait complete. 2024-03-08T08:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.790+0000 7f5be71b6640 1 Processor -- start 2024-03-08T08:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.790+0000 7f5be71b6640 1 -- start start 2024-03-08T08:30:03.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5be71b6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be0072440 0x7f5be00cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5be71b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00b4d00 0x7f5be0151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5be71b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be015b4a0 0x7f5be01503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5be71b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be00729a0 con 0x7f5be00b4d00 2024-03-08T08:30:03.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5be71b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0072820 con 0x7f5be015b4a0 2024-03-08T08:30:03.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5be71b6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be006e7b0 con 0x7f5be0072440 2024-03-08T08:30:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00b4d00 0x7f5be0151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00b4d00 0x7f5be0151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44728/0 (socket says 172.21.15.105:44728) 2024-03-08T08:30:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 -- 172.21.15.105:0/423691817 learned_addr learned my addr 172.21.15.105:0/423691817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5be4f52640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be0072440 0x7f5be00cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5be5753640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be015b4a0 0x7f5be01503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 -- 172.21.15.105:0/423691817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be0072440 msgr2=0x7f5be00cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be0072440 0x7f5be00cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 -- 172.21.15.105:0/423691817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be015b4a0 msgr2=0x7f5be01503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be015b4a0 0x7f5be01503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 -- 172.21.15.105:0/423691817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0152530 con 0x7f5be00b4d00 2024-03-08T08:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdffff640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00b4d00 0x7f5be0151ff0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5bd402bee0 tx=0x7f5bd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.791+0000 7f5bdd7fa640 1 -- 172.21.15.105:0/423691817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4035040 con 0x7f5be00b4d00 2024-03-08T08:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.792+0000 7f5bdd7fa640 1 -- 172.21.15.105:0/423691817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd403b920 con 0x7f5be00b4d00 2024-03-08T08:30:03.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.792+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be01551d0 con 0x7f5be00b4d00 2024-03-08T08:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.792+0000 7f5bdd7fa640 1 -- 172.21.15.105:0/423691817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd403bd10 con 0x7f5be00b4d00 2024-03-08T08:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.792+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be0154c10 con 0x7f5be00b4d00 2024-03-08T08:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.793+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be0075120 con 0x7f5be00b4d00 2024-03-08T08:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.793+0000 7f5bdd7fa640 1 -- 172.21.15.105:0/423691817 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bd4038020 con 0x7f5be00b4d00 2024-03-08T08:30:03.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.793+0000 7f5bdd7fa640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5bcc06b500 0x7f5bcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.794+0000 7f5bdd7fa640 1 -- 172.21.15.105:0/423691817 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5bd40abed0 con 0x7f5be00b4d00 2024-03-08T08:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.794+0000 7f5be4f52640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5bcc06b500 0x7f5bcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.794+0000 7f5be4f52640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5bcc06b500 0x7f5bcc06d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009bd0 tx=0x7f5bd800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.798+0000 7f5bdd7fa640 1 -- 172.21.15.105:0/423691817 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bd407aa00 con 0x7f5be00b4d00 2024-03-08T08:30:03.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.966+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5be0151890 con 0x7f5bcc06b500 2024-03-08T08:30:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.966+0000 7f5bdd7fa640 1 -- 172.21.15.105:0/423691817 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5be0151890 con 0x7f5bcc06b500 2024-03-08T08:30:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 0 to deep-scrub 2024-03-08T08:30:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5bcc06b500 msgr2=0x7f5bcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5bcc06b500 0x7f5bcc06d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009bd0 tx=0x7f5bd800b040 comp rx=0 tx=0).stop 2024-03-08T08:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00b4d00 msgr2=0x7f5be0151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00b4d00 0x7f5be0151ff0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5bd402bee0 tx=0x7f5bd40046c0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 shutdown_connections 2024-03-08T08:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5bcc06b500 0x7f5bcc06d9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be015b4a0 0x7f5be01503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5be00b4d00 0x7f5be0151ff0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 --2- 172.21.15.105:0/423691817 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be0072440 0x7f5be00cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 >> 172.21.15.105:0/423691817 conn(0x7f5be00b1740 msgr2=0x7f5be006e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 shutdown_connections 2024-03-08T08:30:03.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:02.968+0000 7f5be71b6640 1 -- 172.21.15.105:0/423691817 wait complete. 2024-03-08T08:30:03.145 INFO:tasks.ceph:Scrubbing osd.1 2024-03-08T08:30:03.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-08T08:30:03.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.248+0000 7fc5170df640 1 Processor -- start 2024-03-08T08:30:03.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.249+0000 7fc5170df640 1 -- start start 2024-03-08T08:30:03.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.249+0000 7fc5170df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51015be50 0x7fc5100b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.249+0000 7fc5170df640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5100ceac0 0x7fc5100cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.249+0000 7fc5170df640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5100cd580 con 0x7fc5100cd0f0 2024-03-08T08:30:03.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.249+0000 7fc5170df640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5100ce070 con 0x7fc5100ceac0 2024-03-08T08:30:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.249+0000 7fc5170df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100cd860 con 0x7fc51015be50 2024-03-08T08:30:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc514e7b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51015be50 0x7fc5100b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc51567c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5100ceac0 0x7fc5100cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc51567c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc5100ceac0 0x7fc5100cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49382/0 (socket says 172.21.15.105:49382) 2024-03-08T08:30:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc514e7b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51015be50 0x7fc5100b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.105:58526/0 (socket says 172.21.15.105:58526) 2024-03-08T08:30:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc51567c640 1 -- 172.21.15.105:0/1204982269 learned_addr learned my addr 172.21.15.105:0/1204982269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc514e7b640 1 -- 172.21.15.105:0/1204982269 >> v2:172.21.15.144:3300/0 conn(0x7fc5100ceac0 msgr2=0x7fc5100cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:03.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc514e7b640 1 --2- 172.21.15.105:0/1204982269 >> v2:172.21.15.144:3300/0 conn(0x7fc5100ceac0 0x7fc5100cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc514e7b640 1 -- 172.21.15.105:0/1204982269 >> v1:172.21.15.144:6789/0 conn(0x7fc5100cd0f0 legacy=0x7fc51006e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc514e7b640 1 -- 172.21.15.105:0/1204982269 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100ce2c0 con 0x7fc51015be50 2024-03-08T08:30:03.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.250+0000 7fc51567c640 1 --2- 172.21.15.105:0/1204982269 >> v2:172.21.15.144:3300/0 conn(0x7fc5100ceac0 0x7fc5100cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:30:03.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc514e7b640 1 --2- 172.21.15.105:0/1204982269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51015be50 0x7fc5100b40b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc5040097a0 tx=0x7fc50402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa18a123c596bc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc50f7fe640 1 -- 172.21.15.105:0/1204982269 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5040315a0 con 0x7fc51015be50 2024-03-08T08:30:03.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc50f7fe640 1 -- 172.21.15.105:0/1204982269 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc504037070 con 0x7fc51015be50 2024-03-08T08:30:03.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc50f7fe640 1 -- 172.21.15.105:0/1204982269 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504031960 con 0x7fc51015be50 2024-03-08T08:30:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc5170df640 1 -- 172.21.15.105:0/1204982269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51015be50 msgr2=0x7fc5100b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc5170df640 1 --2- 172.21.15.105:0/1204982269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51015be50 0x7fc5100b40b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc5040097a0 tx=0x7fc50402b840 comp rx=0 tx=0).stop 2024-03-08T08:30:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc5170df640 1 -- 172.21.15.105:0/1204982269 shutdown_connections 2024-03-08T08:30:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc5170df640 1 --2- 172.21.15.105:0/1204982269 >> v2:172.21.15.144:3300/0 conn(0x7fc5100ceac0 0x7fc5100cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc5170df640 1 --2- 172.21.15.105:0/1204982269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc51015be50 0x7fc5100b40b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc5170df640 1 -- 172.21.15.105:0/1204982269 >> 172.21.15.105:0/1204982269 conn(0x7fc5100b3590 msgr2=0x7fc5100b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.251+0000 7fc5170df640 1 -- 172.21.15.105:0/1204982269 shutdown_connections 2024-03-08T08:30:03.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.252+0000 7fc5170df640 1 -- 172.21.15.105:0/1204982269 wait complete. 2024-03-08T08:30:03.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.253+0000 7fc5170df640 1 Processor -- start 2024-03-08T08:30:03.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.253+0000 7fc5170df640 1 -- start start 2024-03-08T08:30:03.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.253+0000 7fc5170df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5100ceac0 0x7fc5101f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.253+0000 7fc5170df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51015be50 0x7fc5100b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.253+0000 7fc5170df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5101f20c0 0x7fc5101f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.253+0000 7fc5170df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51006f0a0 con 0x7fc5100ceac0 2024-03-08T08:30:03.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.253+0000 7fc5170df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100cf9e0 con 0x7fc51015be50 2024-03-08T08:30:03.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.253+0000 7fc5170df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5101f2a80 con 0x7fc5101f20c0 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5100ceac0 0x7fc5101f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5100ceac0 0x7fc5101f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44744/0 (socket says 172.21.15.105:44744) 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 -- 172.21.15.105:0/3879642454 learned_addr learned my addr 172.21.15.105:0/3879642454 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc51567c640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5101f20c0 0x7fc5101f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc50ffff640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51015be50 0x7fc5100b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 -- 172.21.15.105:0/3879642454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5101f20c0 msgr2=0x7fc5101f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5101f20c0 0x7fc5101f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 -- 172.21.15.105:0/3879642454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51015be50 msgr2=0x7fc5100b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc51015be50 0x7fc5100b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 -- 172.21.15.105:0/3879642454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51015b310 con 0x7fc5100ceac0 2024-03-08T08:30:03.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc514e7b640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5100ceac0 0x7fc5101f6210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc50402be50 tx=0x7fc5040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504032630 con 0x7fc5100ceac0 2024-03-08T08:30:03.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5040327f0 con 0x7fc5100ceac0 2024-03-08T08:30:03.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5101f2ca0 con 0x7fc5100ceac0 2024-03-08T08:30:03.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504032be0 con 0x7fc5100ceac0 2024-03-08T08:30:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.254+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5101f5a10 con 0x7fc5100ceac0 2024-03-08T08:30:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.255+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc510075200 con 0x7fc5100ceac0 2024-03-08T08:30:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.256+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc50403b3f0 con 0x7fc5100ceac0 2024-03-08T08:30:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.256+0000 7fc50d7fa640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4f406b4d0 0x7fc4f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.256+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc50402f030 con 0x7fc5100ceac0 2024-03-08T08:30:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.256+0000 7fc50ffff640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4f406b4d0 0x7fc4f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.256+0000 7fc50d7fa640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7fc4f4071390 0x7fc4f40737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.256+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 --> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc504043ec0 con 0x7fc4f4071390 2024-03-08T08:30:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.256+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5040b02c0 con 0x7fc5100ceac0 2024-03-08T08:30:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.256+0000 7fc51567c640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7fc4f4071390 0x7fc4f40737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.257+0000 7fc50ffff640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4f406b4d0 0x7fc4f406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5100cb240 tx=0x7fc4fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.257+0000 7fc51567c640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7fc4f4071390 0x7fc4f40737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.257+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 <== osd.1 v2:172.21.15.105:6824/2989492650 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc504043ec0 con 0x7fc4f4071390 2024-03-08T08:30:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.271+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 --> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc5100b79c0 con 0x7fc4f4071390 2024-03-08T08:30:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.271+0000 7fc50d7fa640 1 -- 172.21.15.105:0/3879642454 <== osd.1 v2:172.21.15.105:6824/2989492650 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc5100b79c0 con 0x7fc4f4071390 2024-03-08T08:30:03.273 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:30:03.273 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T08:30:03.273 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:30:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7fc4f4071390 msgr2=0x7fc4f40737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6824/2989492650,v1:172.21.15.105:6825/2989492650] conn(0x7fc4f4071390 0x7fc4f40737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4f406b4d0 msgr2=0x7fc4f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fc4f406b4d0 0x7fc4f406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5100cb240 tx=0x7fc4fc009040 comp rx=0 tx=0).stop 2024-03-08T08:30:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5100ceac0 msgr2=0x7fc5101f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 --2- 172.21.15.105:0/3879642454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5100ceac0 0x7fc5101f6210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc50402be50 tx=0x7fc5040046c0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc51567c640 1 -- 172.21.15.105:0/3879642454 reap_dead start 2024-03-08T08:30:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 shutdown_connections 2024-03-08T08:30:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 >> 172.21.15.105:0/3879642454 conn(0x7fc5100b3590 msgr2=0x7fc5100b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 shutdown_connections 2024-03-08T08:30:03.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.272+0000 7fc5170df640 1 -- 172.21.15.105:0/3879642454 wait complete. 2024-03-08T08:30:03.283 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-08T08:30:03.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.384+0000 7f5fe1ec3640 1 Processor -- start 2024-03-08T08:30:03.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.384+0000 7f5fe1ec3640 1 -- start start 2024-03-08T08:30:03.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fe1ec3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fe1ec3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5fdc1584b0 0x7f5fdc15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fe1ec3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fdc0af8b0 con 0x7f5fdc0b3260 2024-03-08T08:30:03.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fe1ec3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc0ad910 con 0x7f5fdc1584b0 2024-03-08T08:30:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fe1ec3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0adbf0 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdbfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5fdc0b3260 0x7f5fdc161850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:51908/0 (socket says 172.21.15.105:51908) 2024-03-08T08:30:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdbfff640 1 -- 172.21.15.105:0/3000702881 learned_addr learned my addr 172.21.15.105:0/3000702881 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdb7fe640 1 --2- 172.21.15.105:0/3000702881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fd2ffd640 1 --2- 172.21.15.105:0/3000702881 >> v2:172.21.15.144:3300/0 conn(0x7f5fdc1584b0 0x7f5fdc15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdaffd640 1 -- 172.21.15.105:0/3000702881 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549146090 0 0) 0x7f5fdc0af8b0 con 0x7f5fdc0b3260 2024-03-08T08:30:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdaffd640 1 -- 172.21.15.105:0/3000702881 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc8003520 con 0x7f5fdc0b3260 2024-03-08T08:30:03.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdb7fe640 1 -- 172.21.15.105:0/3000702881 >> v2:172.21.15.144:3300/0 conn(0x7f5fdc1584b0 msgr2=0x7f5fdc15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdb7fe640 1 --2- 172.21.15.105:0/3000702881 >> v2:172.21.15.144:3300/0 conn(0x7f5fdc1584b0 0x7f5fdc15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdb7fe640 1 -- 172.21.15.105:0/3000702881 >> v1:172.21.15.105:6789/0 conn(0x7f5fdc0b3260 legacy=0x7f5fdc161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.385+0000 7f5fdb7fe640 1 -- 172.21.15.105:0/3000702881 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0aded0 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.386+0000 7f5fdb7fe640 1 --2- 172.21.15.105:0/3000702881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc0b4d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0097a0 tx=0x7f5fcc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12d4fc146ec89e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.386+0000 7f5fdaffd640 1 -- 172.21.15.105:0/3000702881 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc035020 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.386+0000 7f5fdaffd640 1 -- 172.21.15.105:0/3000702881 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc031830 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.386+0000 7f5fdaffd640 1 -- 172.21.15.105:0/3000702881 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc031be0 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.387+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/3000702881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 msgr2=0x7f5fdc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.387+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/3000702881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc0b4d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0097a0 tx=0x7f5fcc02b8d0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.387+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/3000702881 shutdown_connections 2024-03-08T08:30:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.387+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/3000702881 >> v2:172.21.15.144:3300/0 conn(0x7f5fdc1584b0 0x7f5fdc15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.387+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/3000702881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc0b4d50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.387+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/3000702881 >> 172.21.15.105:0/3000702881 conn(0x7f5fdc0cfdd0 msgr2=0x7f5fdc0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.387+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/3000702881 shutdown_connections 2024-03-08T08:30:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.387+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/3000702881 wait complete. 2024-03-08T08:30:03.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.388+0000 7f5fe1ec3640 1 Processor -- start 2024-03-08T08:30:03.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.388+0000 7f5fe1ec3640 1 -- start start 2024-03-08T08:30:03.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fe1ec3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fe1ec3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc0b3260 0x7f5fdc145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fe1ec3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fdc1584b0 0x7f5fdc142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fe1ec3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0b36f0 con 0x7f5fdc1584b0 2024-03-08T08:30:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fe1ec3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0ae490 con 0x7f5fdc0b3260 2024-03-08T08:30:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fe1ec3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0ae9b0 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fdc1584b0 0x7f5fdc142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fdc1584b0 0x7f5fdc142ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44746/0 (socket says 172.21.15.105:44746) 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdbfff640 1 -- 172.21.15.105:0/4172494735 learned_addr learned my addr 172.21.15.105:0/4172494735 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fd2ffd640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc0b3260 0x7f5fdc145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdb7fe640 1 -- 172.21.15.105:0/4172494735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc0b3260 msgr2=0x7f5fdc145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdb7fe640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc0b3260 0x7f5fdc145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdb7fe640 1 -- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fdc1584b0 msgr2=0x7f5fdc142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdb7fe640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fdc1584b0 0x7f5fdc142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdb7fe640 1 -- 172.21.15.105:0/4172494735 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc143010 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.389+0000 7f5fdbfff640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fdc1584b0 0x7f5fdc142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:30:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.390+0000 7f5fdb7fe640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc143320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc033000 tx=0x7f5fcc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.390+0000 7f5fd8ff9640 1 -- 172.21.15.105:0/4172494735 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc048070 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.390+0000 7f5fd8ff9640 1 -- 172.21.15.105:0/4172494735 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc03a630 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.390+0000 7f5fd8ff9640 1 -- 172.21.15.105:0/4172494735 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc03aa00 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.390+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc143a50 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.390+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc144010 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.391+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc075120 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.391+0000 7f5fd8ff9640 1 -- 172.21.15.105:0/4172494735 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fcc038020 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.392+0000 7f5fd8ff9640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5fbc06b4d0 0x7f5fbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.392+0000 7f5fd8ff9640 1 -- 172.21.15.105:0/4172494735 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5fcc0ab250 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.392+0000 7f5fd2ffd640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5fbc06b4d0 0x7f5fbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.392+0000 7f5fd2ffd640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5fbc06b4d0 0x7f5fbc06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc152220 tx=0x7f5fc4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.400+0000 7f5fd8ff9640 1 -- 172.21.15.105:0/4172494735 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fcc079ce0 con 0x7f5fdc0b2e80 2024-03-08T08:30:03.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.569+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fdc15dc80 con 0x7f5fbc06b4d0 2024-03-08T08:30:03.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.569+0000 7f5fd8ff9640 1 -- 172.21.15.105:0/4172494735 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5fdc15dc80 con 0x7f5fbc06b4d0 2024-03-08T08:30:03.571 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 1 to deep-scrub 2024-03-08T08:30:03.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5fbc06b4d0 msgr2=0x7f5fbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5fbc06b4d0 0x7f5fbc06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc152220 tx=0x7f5fc4009190 comp rx=0 tx=0).stop 2024-03-08T08:30:03.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 msgr2=0x7f5fdc143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc143320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc033000 tx=0x7f5fcc0046c0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 shutdown_connections 2024-03-08T08:30:03.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5fbc06b4d0 0x7f5fbc06d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fdc1584b0 0x7f5fdc142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fdc0b3260 0x7f5fdc145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 --2- 172.21.15.105:0/4172494735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fdc0b2e80 0x7f5fdc143320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 >> 172.21.15.105:0/4172494735 conn(0x7f5fdc0cfdd0 msgr2=0x7f5fdc0cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 shutdown_connections 2024-03-08T08:30:03.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.571+0000 7f5fe1ec3640 1 -- 172.21.15.105:0/4172494735 wait complete. 2024-03-08T08:30:03.583 INFO:tasks.ceph:Scrubbing osd.2 2024-03-08T08:30:03.584 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-08T08:30:03.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.686+0000 7fd22bbb5640 1 Processor -- start 2024-03-08T08:30:03.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.686+0000 7fd22bbb5640 1 -- start start 2024-03-08T08:30:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.687+0000 7fd22bbb5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd2240b5b80 0x7fd2240cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.687+0000 7fd22bbb5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2240ce2a0 0x7fd22415e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.687+0000 7fd22bbb5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22406dc30 con 0x7fd224144420 2024-03-08T08:30:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.687+0000 7fd22bbb5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22406df10 con 0x7fd2240ce2a0 2024-03-08T08:30:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.687+0000 7fd22bbb5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22406e1f0 con 0x7fd2240b5b80 2024-03-08T08:30:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.687+0000 7fd229951640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd224144420 0x7fd22406d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55792/0 (socket says 172.21.15.105:55792) 2024-03-08T08:30:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.687+0000 7fd229951640 1 -- 172.21.15.105:0/3937891437 learned_addr learned my addr 172.21.15.105:0/3937891437 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.687+0000 7fd22a152640 1 --2- 172.21.15.105:0/3937891437 >> v2:172.21.15.105:3300/0 conn(0x7fd2240ce2a0 0x7fd22415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.688+0000 7fd229150640 1 --2- 172.21.15.105:0/3937891437 >> v2:172.21.15.144:3300/0 conn(0x7fd2240b5b80 0x7fd2240cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.688+0000 7fd22894f640 1 -- 172.21.15.105:0/3937891437 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332163767 0 0) 0x7fd22406dc30 con 0x7fd224144420 2024-03-08T08:30:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.688+0000 7fd22894f640 1 -- 172.21.15.105:0/3937891437 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210003520 con 0x7fd224144420 2024-03-08T08:30:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.688+0000 7fd22a152640 1 -- 172.21.15.105:0/3937891437 >> v2:172.21.15.144:3300/0 conn(0x7fd2240b5b80 msgr2=0x7fd2240cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.688+0000 7fd22a152640 1 --2- 172.21.15.105:0/3937891437 >> v2:172.21.15.144:3300/0 conn(0x7fd2240b5b80 0x7fd2240cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.688+0000 7fd22a152640 1 -- 172.21.15.105:0/3937891437 >> v1:172.21.15.105:6789/0 conn(0x7fd224144420 legacy=0x7fd22406d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.688+0000 7fd22a152640 1 -- 172.21.15.105:0/3937891437 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22406e4d0 con 0x7fd2240ce2a0 2024-03-08T08:30:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.688+0000 7fd22a152640 1 --2- 172.21.15.105:0/3937891437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 0x7fd22415e830 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd224074690 tx=0x7fd21c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e5273d6b549a19 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22894f640 1 -- 172.21.15.105:0/3937891437 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c039070 con 0x7fd2240ce2a0 2024-03-08T08:30:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22894f640 1 -- 172.21.15.105:0/3937891437 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21c02cc70 con 0x7fd2240ce2a0 2024-03-08T08:30:03.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22894f640 1 -- 172.21.15.105:0/3937891437 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c034660 con 0x7fd2240ce2a0 2024-03-08T08:30:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3937891437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 msgr2=0x7fd22415e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22bbb5640 1 --2- 172.21.15.105:0/3937891437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 0x7fd22415e830 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd224074690 tx=0x7fd21c02bec0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3937891437 shutdown_connections 2024-03-08T08:30:03.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22bbb5640 1 --2- 172.21.15.105:0/3937891437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 0x7fd22415e830 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22bbb5640 1 --2- 172.21.15.105:0/3937891437 >> v2:172.21.15.144:3300/0 conn(0x7fd2240b5b80 0x7fd2240cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3937891437 >> 172.21.15.105:0/3937891437 conn(0x7fd2240d09e0 msgr2=0x7fd2240d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3937891437 shutdown_connections 2024-03-08T08:30:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.689+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3937891437 wait complete. 2024-03-08T08:30:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.690+0000 7fd22bbb5640 1 Processor -- start 2024-03-08T08:30:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.690+0000 7fd22bbb5640 1 -- start start 2024-03-08T08:30:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd22bbb5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240b5b80 0x7fd2240b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd22bbb5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 0x7fd224137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd22bbb5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd224144420 0x7fd22413aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd22bbb5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22415ed70 con 0x7fd2240ce2a0 2024-03-08T08:30:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd22bbb5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd224144800 con 0x7fd2240b5b80 2024-03-08T08:30:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd22bbb5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22415eef0 con 0x7fd224144420 2024-03-08T08:30:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd229150640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 0x7fd224137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd229150640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 0x7fd224137fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38220/0 (socket says 172.21.15.105:38220) 2024-03-08T08:30:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd229150640 1 -- 172.21.15.105:0/3199354331 learned_addr learned my addr 172.21.15.105:0/3199354331 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd22a152640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd224144420 0x7fd22413aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd229150640 1 -- 172.21.15.105:0/3199354331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd224144420 msgr2=0x7fd22413aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd229150640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd224144420 0x7fd22413aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd229150640 1 -- 172.21.15.105:0/3199354331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240b5b80 msgr2=0x7fd2240b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.691+0000 7fd229951640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240b5b80 0x7fd2240b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.692+0000 7fd229150640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2240b5b80 0x7fd2240b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.692+0000 7fd229150640 1 -- 172.21.15.105:0/3199354331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22413ca70 con 0x7fd2240ce2a0 2024-03-08T08:30:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.692+0000 7fd229150640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 0x7fd224137fc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd21400d800 tx=0x7fd21400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.692+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400b6c0 con 0x7fd2240ce2a0 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.692+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214019070 con 0x7fd2240ce2a0 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.692+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400ba80 con 0x7fd2240ce2a0 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.692+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2240b1830 con 0x7fd2240ce2a0 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.692+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22413ccf0 con 0x7fd2240ce2a0 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.693+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd224075200 con 0x7fd2240ce2a0 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.693+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd21400bc20 con 0x7fd2240ce2a0 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.693+0000 7fd21a7fc640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20806b4d0 0x7fd20806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.694+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd21408a350 con 0x7fd2240ce2a0 2024-03-08T08:30:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.694+0000 7fd229951640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20806b4d0 0x7fd20806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.694+0000 7fd21a7fc640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fd208071390 0x7fd2080737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.694+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 --> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd214017e50 con 0x7fd208071390 2024-03-08T08:30:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.694+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd214091050 con 0x7fd2240ce2a0 2024-03-08T08:30:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.694+0000 7fd22a152640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fd208071390 0x7fd2080737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.694+0000 7fd229951640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20806b4d0 0x7fd20806d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd220000f60 tx=0x7fd220009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.694+0000 7fd22a152640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fd208071390 0x7fd2080737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.695+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 <== osd.2 v2:172.21.15.105:6800/1086051235 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd214017e50 con 0x7fd208071390 2024-03-08T08:30:03.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 --> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd2240b1d50 con 0x7fd208071390 2024-03-08T08:30:03.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd21a7fc640 1 -- 172.21.15.105:0/3199354331 <== osd.2 v2:172.21.15.105:6800/1086051235 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd2240b1d50 con 0x7fd208071390 2024-03-08T08:30:03.709 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:30:03.709 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T08:30:03.709 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:30:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fd208071390 msgr2=0x7fd2080737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6800/1086051235,v1:172.21.15.105:6801/1086051235] conn(0x7fd208071390 0x7fd2080737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20806b4d0 msgr2=0x7fd20806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd20806b4d0 0x7fd20806d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd220000f60 tx=0x7fd220009040 comp rx=0 tx=0).stop 2024-03-08T08:30:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 msgr2=0x7fd224137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 --2- 172.21.15.105:0/3199354331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2240ce2a0 0x7fd224137fc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd21400d800 tx=0x7fd21400dcc0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22a152640 1 -- 172.21.15.105:0/3199354331 reap_dead start 2024-03-08T08:30:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 shutdown_connections 2024-03-08T08:30:03.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 >> 172.21.15.105:0/3199354331 conn(0x7fd2240d09e0 msgr2=0x7fd22406d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 shutdown_connections 2024-03-08T08:30:03.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.708+0000 7fd22bbb5640 1 -- 172.21.15.105:0/3199354331 wait complete. 2024-03-08T08:30:03.719 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-08T08:30:03.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.819+0000 7f53716da640 1 Processor -- start 2024-03-08T08:30:03.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.819+0000 7f53716da640 1 -- start start 2024-03-08T08:30:03.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f53716da640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f536c0d2530 0x7f536c0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f53716da640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f53716da640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c0b3880 con 0x7f536c0d2910 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f53716da640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0d48f0 con 0x7f536c0d2530 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f53716da640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0d4bb0 con 0x7f536c0d43e0 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f536a7fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f536b7fe640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f536c0d2910 0x7f536c15de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.105:38954/0 (socket says 172.21.15.105:38954) 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f536affd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f536c0d2530 0x7f536c0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.820+0000 7f536b7fe640 1 -- 172.21.15.105:0/2281615387 learned_addr learned my addr 172.21.15.105:0/2281615387 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.821+0000 7f536a7fc640 1 -- 172.21.15.105:0/2281615387 >> v2:172.21.15.144:3300/0 conn(0x7f536c0d2530 msgr2=0x7f536c0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.821+0000 7f536a7fc640 1 --2- 172.21.15.105:0/2281615387 >> v2:172.21.15.144:3300/0 conn(0x7f536c0d2530 0x7f536c0b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.821+0000 7f536a7fc640 1 -- 172.21.15.105:0/2281615387 >> v1:172.21.15.144:6789/0 conn(0x7f536c0d2910 legacy=0x7f536c15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.821+0000 7f536a7fc640 1 -- 172.21.15.105:0/2281615387 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0ceb30 con 0x7f536c0d43e0 2024-03-08T08:30:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.821+0000 7f536affd640 1 --2- 172.21.15.105:0/2281615387 >> v2:172.21.15.144:3300/0 conn(0x7f536c0d2530 0x7f536c0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T08:30:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.821+0000 7f536a7fc640 1 --2- 172.21.15.105:0/2281615387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c0b3340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53600097a0 tx=0x7f536002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159ede990bd4fb0e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.821+0000 7f5369ffb640 1 -- 172.21.15.105:0/2281615387 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5360038070 con 0x7f536c0d43e0 2024-03-08T08:30:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f5369ffb640 1 -- 172.21.15.105:0/2281615387 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5360031700 con 0x7f536c0d43e0 2024-03-08T08:30:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f5369ffb640 1 -- 172.21.15.105:0/2281615387 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5360031ad0 con 0x7f536c0d43e0 2024-03-08T08:30:03.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f53716da640 1 -- 172.21.15.105:0/2281615387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 msgr2=0x7f536c0b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f53716da640 1 --2- 172.21.15.105:0/2281615387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c0b3340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53600097a0 tx=0x7f536002b7a0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f53716da640 1 -- 172.21.15.105:0/2281615387 shutdown_connections 2024-03-08T08:30:03.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f53716da640 1 --2- 172.21.15.105:0/2281615387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c0b3340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f53716da640 1 --2- 172.21.15.105:0/2281615387 >> v2:172.21.15.144:3300/0 conn(0x7f536c0d2530 0x7f536c0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f53716da640 1 -- 172.21.15.105:0/2281615387 >> 172.21.15.105:0/2281615387 conn(0x7f536c09cfb0 msgr2=0x7f536c0d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:03.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f53716da640 1 -- 172.21.15.105:0/2281615387 shutdown_connections 2024-03-08T08:30:03.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.822+0000 7f53716da640 1 -- 172.21.15.105:0/2281615387 wait complete. 2024-03-08T08:30:03.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.823+0000 7f53716da640 1 Processor -- start 2024-03-08T08:30:03.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.823+0000 7f53716da640 1 -- start start 2024-03-08T08:30:03.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f53716da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f536c0d2530 0x7f536c09da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f53716da640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f536c0d2910 0x7f536c1f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f53716da640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f53716da640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0b3080 con 0x7f536c0d2530 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f53716da640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0cf0f0 con 0x7f536c0d2910 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f53716da640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c15e510 con 0x7f536c0d43e0 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f536affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f536c0d2530 0x7f536c09da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f536affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f536c0d2530 0x7f536c09da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38232/0 (socket says 172.21.15.105:38232) 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f536affd640 1 -- 172.21.15.105:0/90844522 learned_addr learned my addr 172.21.15.105:0/90844522 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:03.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f536b7fe640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.824+0000 7f536a7fc640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f536c0d2910 0x7f536c1f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f536affd640 1 -- 172.21.15.105:0/90844522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 msgr2=0x7f536c1f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f536affd640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c1f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f536affd640 1 -- 172.21.15.105:0/90844522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f536c0d2910 msgr2=0x7f536c1f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f536affd640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f536c0d2910 0x7f536c1f3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f536affd640 1 -- 172.21.15.105:0/90844522 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0a1450 con 0x7f536c0d2530 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f536affd640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f536c0d2530 0x7f536c09da90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ebe0 tx=0x7f535c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f53537fe640 1 -- 172.21.15.105:0/90844522 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c017070 con 0x7f536c0d2530 2024-03-08T08:30:03.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f53537fe640 1 -- 172.21.15.105:0/90844522 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c00cc30 con 0x7f536c0d2530 2024-03-08T08:30:03.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f53537fe640 1 -- 172.21.15.105:0/90844522 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c013470 con 0x7f536c0d2530 2024-03-08T08:30:03.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c0cbed0 con 0x7f536c0d2530 2024-03-08T08:30:03.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.825+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c0cc490 con 0x7f536c0d2530 2024-03-08T08:30:03.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.826+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c075120 con 0x7f536c0d2530 2024-03-08T08:30:03.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.827+0000 7f53537fe640 1 -- 172.21.15.105:0/90844522 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f535c01a070 con 0x7f536c0d2530 2024-03-08T08:30:03.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.827+0000 7f53537fe640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f535806b500 0x7f535806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:03.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.827+0000 7f53537fe640 1 -- 172.21.15.105:0/90844522 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f535c089d00 con 0x7f536c0d2530 2024-03-08T08:30:03.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.827+0000 7f536a7fc640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f535806b500 0x7f535806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:03.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.827+0000 7f536a7fc640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f535806b500 0x7f535806d9c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5360035000 tx=0x7f5360008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:03.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:03.831+0000 7f53537fe640 1 -- 172.21.15.105:0/90844522 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f535c0588c0 con 0x7f536c0d2530 2024-03-08T08:30:04.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.001+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f536c0d3e50 con 0x7f535806b500 2024-03-08T08:30:04.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.001+0000 7f53537fe640 1 -- 172.21.15.105:0/90844522 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f536c0d3e50 con 0x7f535806b500 2024-03-08T08:30:04.003 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 2 to deep-scrub 2024-03-08T08:30:04.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f535806b500 msgr2=0x7f535806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f535806b500 0x7f535806d9c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5360035000 tx=0x7f5360008030 comp rx=0 tx=0).stop 2024-03-08T08:30:04.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f536c0d2530 msgr2=0x7f536c09da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f536c0d2530 0x7f536c09da90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ebe0 tx=0x7f535c00c520 comp rx=0 tx=0).stop 2024-03-08T08:30:04.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 shutdown_connections 2024-03-08T08:30:04.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f535806b500 0x7f535806d9c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f536c0d43e0 0x7f536c1f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f536c0d2910 0x7f536c1f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 --2- 172.21.15.105:0/90844522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f536c0d2530 0x7f536c09da90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 >> 172.21.15.105:0/90844522 conn(0x7f536c09cfb0 msgr2=0x7f536c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 shutdown_connections 2024-03-08T08:30:04.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.003+0000 7f53716da640 1 -- 172.21.15.105:0/90844522 wait complete. 2024-03-08T08:30:04.015 INFO:tasks.ceph:Scrubbing osd.3 2024-03-08T08:30:04.015 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-08T08:30:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.118+0000 7ff78f695640 1 Processor -- start 2024-03-08T08:30:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.118+0000 7ff78f695640 1 -- start start 2024-03-08T08:30:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78f695640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7881614c0 0x7ff7880b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78f695640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881618a0 0x7ff7880b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78f695640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7880b5310 con 0x7ff7880d47e0 2024-03-08T08:30:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78f695640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7880d4c70 con 0x7ff7881614c0 2024-03-08T08:30:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78f695640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880d1890 con 0x7ff7881618a0 2024-03-08T08:30:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78d431640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7881614c0 0x7ff7880b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78d431640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7881614c0 0x7ff7880b6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38236/0 (socket says 172.21.15.105:38236) 2024-03-08T08:30:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78d431640 1 -- 172.21.15.105:0/1863763693 learned_addr learned my addr 172.21.15.105:0/1863763693 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.119+0000 7ff78cc30640 1 --2- 172.21.15.105:0/1863763693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881618a0 0x7ff7880b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.120+0000 7ff78d431640 1 -- 172.21.15.105:0/1863763693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881618a0 msgr2=0x7ff7880b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.120+0000 7ff78d431640 1 --2- 172.21.15.105:0/1863763693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881618a0 0x7ff7880b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.120+0000 7ff78d431640 1 -- 172.21.15.105:0/1863763693 >> v1:172.21.15.144:6789/0 conn(0x7ff7880d47e0 legacy=0x7ff7880d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.120+0000 7ff78d431640 1 -- 172.21.15.105:0/1863763693 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880cf5c0 con 0x7ff7881614c0 2024-03-08T08:30:04.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.120+0000 7ff78d431640 1 --2- 172.21.15.105:0/1863763693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881614c0 0x7ff7880b6b10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff784009870 tx=0x7ff78402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0a742b687b3ae9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.120+0000 7ff77ffff640 1 -- 172.21.15.105:0/1863763693 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784034070 con 0x7ff7881614c0 2024-03-08T08:30:04.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.120+0000 7ff77ffff640 1 -- 172.21.15.105:0/1863763693 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff784030840 con 0x7ff7881614c0 2024-03-08T08:30:04.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.120+0000 7ff77ffff640 1 -- 172.21.15.105:0/1863763693 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784030c10 con 0x7ff7881614c0 2024-03-08T08:30:04.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.121+0000 7ff78f695640 1 -- 172.21.15.105:0/1863763693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881614c0 msgr2=0x7ff7880b6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.121+0000 7ff78f695640 1 --2- 172.21.15.105:0/1863763693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881614c0 0x7ff7880b6b10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff784009870 tx=0x7ff78402b980 comp rx=0 tx=0).stop 2024-03-08T08:30:04.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.121+0000 7ff78f695640 1 -- 172.21.15.105:0/1863763693 shutdown_connections 2024-03-08T08:30:04.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.121+0000 7ff78f695640 1 --2- 172.21.15.105:0/1863763693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881618a0 0x7ff7880b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.121+0000 7ff78f695640 1 --2- 172.21.15.105:0/1863763693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881614c0 0x7ff7880b6b10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.121+0000 7ff78f695640 1 -- 172.21.15.105:0/1863763693 >> 172.21.15.105:0/1863763693 conn(0x7ff7880d3cc0 msgr2=0x7ff7880d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.121+0000 7ff78f695640 1 -- 172.21.15.105:0/1863763693 shutdown_connections 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.121+0000 7ff78f695640 1 -- 172.21.15.105:0/1863763693 wait complete. 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.122+0000 7ff78f695640 1 Processor -- start 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.122+0000 7ff78f695640 1 -- start start 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78f695640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7880d47e0 0x7ff788155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78f695640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881614c0 0x7ff7880cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78f695640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881618a0 0x7ff788157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78f695640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880d1ae0 con 0x7ff7881618a0 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78f695640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880cfb80 con 0x7ff7880d47e0 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78f695640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880d0dc0 con 0x7ff7881614c0 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881618a0 0x7ff788157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881618a0 0x7ff788157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38250/0 (socket says 172.21.15.105:38250) 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 -- 172.21.15.105:0/103664749 learned_addr learned my addr 172.21.15.105:0/103664749 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78d431640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7880d47e0 0x7ff788155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78cc30640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881614c0 0x7ff7880cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 -- 172.21.15.105:0/103664749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881614c0 msgr2=0x7ff7880cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7881614c0 0x7ff7880cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 -- 172.21.15.105:0/103664749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7880d47e0 msgr2=0x7ff788155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7880d47e0 0x7ff788155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 -- 172.21.15.105:0/103664749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788157590 con 0x7ff7881618a0 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.123+0000 7ff78dc32640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881618a0 0x7ff788157920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff78000c7d0 tx=0x7ff78000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.124+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780003b70 con 0x7ff7881618a0 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.124+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780003d30 con 0x7ff7881618a0 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.124+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff788156cf0 con 0x7ff7881618a0 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.124+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000aa90 con 0x7ff7881618a0 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.124+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7881572b0 con 0x7ff7881618a0 2024-03-08T08:30:04.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.125+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff788075200 con 0x7ff7881618a0 2024-03-08T08:30:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.125+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff78000ac30 con 0x7ff7881618a0 2024-03-08T08:30:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff77dffb640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff76c06b4d0 0x7ff76c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff780089cd0 con 0x7ff7881618a0 2024-03-08T08:30:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff78d431640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff76c06b4d0 0x7ff76c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff77dffb640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7ff76c071390 0x7ff76c0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 --> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff780010e50 con 0x7ff76c071390 2024-03-08T08:30:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff78008c3d0 con 0x7ff7881618a0 2024-03-08T08:30:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff78cc30640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7ff76c071390 0x7ff76c0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff78d431640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff76c06b4d0 0x7ff76c06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff784036000 tx=0x7ff784008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.126+0000 7ff78cc30640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7ff76c071390 0x7ff76c0737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.127+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 <== osd.3 v2:172.21.15.105:6808/2646109234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff780010e50 con 0x7ff76c071390 2024-03-08T08:30:04.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.140+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 --> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff78806ad80 con 0x7ff76c071390 2024-03-08T08:30:04.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.140+0000 7ff77dffb640 1 -- 172.21.15.105:0/103664749 <== osd.3 v2:172.21.15.105:6808/2646109234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff78806ad80 con 0x7ff76c071390 2024-03-08T08:30:04.142 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:30:04.142 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T08:30:04.142 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:30:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7ff76c071390 msgr2=0x7ff76c0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6808/2646109234,v1:172.21.15.105:6809/2646109234] conn(0x7ff76c071390 0x7ff76c0737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff76c06b4d0 msgr2=0x7ff76c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7ff76c06b4d0 0x7ff76c06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff784036000 tx=0x7ff784008030 comp rx=0 tx=0).stop 2024-03-08T08:30:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881618a0 msgr2=0x7ff788157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 --2- 172.21.15.105:0/103664749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7881618a0 0x7ff788157920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff78000c7d0 tx=0x7ff78000cc90 comp rx=0 tx=0).stop 2024-03-08T08:30:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78dc32640 1 -- 172.21.15.105:0/103664749 reap_dead start 2024-03-08T08:30:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 shutdown_connections 2024-03-08T08:30:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 >> 172.21.15.105:0/103664749 conn(0x7ff7880d3cc0 msgr2=0x7ff78806c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 shutdown_connections 2024-03-08T08:30:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.141+0000 7ff78f695640 1 -- 172.21.15.105:0/103664749 wait complete. 2024-03-08T08:30:04.152 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-08T08:30:04.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.251+0000 7fb9b0916640 1 Processor -- start 2024-03-08T08:30:04.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.252+0000 7fb9b0916640 1 -- start start 2024-03-08T08:30:04.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.252+0000 7fb9b0916640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 0x7fb9ac06c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.252+0000 7fb9b0916640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb9ac161850 0x7fb9ac0d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.252+0000 7fb9b0916640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ac0d48a0 con 0x7fb9ac161470 2024-03-08T08:30:04.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.252+0000 7fb9b0916640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac0d2ba0 con 0x7fb9ac161850 2024-03-08T08:30:04.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.252+0000 7fb9b0916640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0d4b80 con 0x7fb9ac06bcb0 2024-03-08T08:30:04.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9aad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb9ac161850 0x7fb9ac0d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9aa59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb9ac161470 0x7fb9ac06b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55800/0 (socket says 172.21.15.105:55800) 2024-03-08T08:30:04.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9aa59c640 1 -- 172.21.15.105:0/477288216 learned_addr learned my addr 172.21.15.105:0/477288216 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:04.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9aad9d640 1 -- 172.21.15.105:0/477288216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 msgr2=0x7fb9ac06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9a9d9b640 1 --2- 172.21.15.105:0/477288216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 0x7fb9ac06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9aad9d640 1 --2- 172.21.15.105:0/477288216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 0x7fb9ac06c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9aad9d640 1 -- 172.21.15.105:0/477288216 >> v1:172.21.15.105:6789/0 conn(0x7fb9ac161470 legacy=0x7fb9ac06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9a959a640 1 -- 172.21.15.105:0/477288216 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381931166 0 0) 0x7fb9ac0d48a0 con 0x7fb9ac161470 2024-03-08T08:30:04.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.253+0000 7fb9aad9d640 1 -- 172.21.15.105:0/477288216 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac06c170 con 0x7fb9ac161850 2024-03-08T08:30:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9aad9d640 1 --2- 172.21.15.105:0/477288216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0d2660 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009d90 tx=0x7fb99c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70750d1167877728 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9a959a640 1 -- 172.21.15.105:0/477288216 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c039070 con 0x7fb9ac161850 2024-03-08T08:30:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9a959a640 1 -- 172.21.15.105:0/477288216 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c02cbc0 con 0x7fb9ac161850 2024-03-08T08:30:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9a959a640 1 -- 172.21.15.105:0/477288216 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c034440 con 0x7fb9ac161850 2024-03-08T08:30:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9b0916640 1 -- 172.21.15.105:0/477288216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 msgr2=0x7fb9ac0d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9b0916640 1 --2- 172.21.15.105:0/477288216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0d2660 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009d90 tx=0x7fb99c02be10 comp rx=0 tx=0).stop 2024-03-08T08:30:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9b0916640 1 -- 172.21.15.105:0/477288216 shutdown_connections 2024-03-08T08:30:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9b0916640 1 --2- 172.21.15.105:0/477288216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0d2660 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9b0916640 1 --2- 172.21.15.105:0/477288216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 0x7fb9ac06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9b0916640 1 -- 172.21.15.105:0/477288216 >> 172.21.15.105:0/477288216 conn(0x7fb9ac0d3ce0 msgr2=0x7fb9ac0d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9b0916640 1 -- 172.21.15.105:0/477288216 shutdown_connections 2024-03-08T08:30:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.254+0000 7fb9b0916640 1 -- 172.21.15.105:0/477288216 wait complete. 2024-03-08T08:30:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9b0916640 1 Processor -- start 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9b0916640 1 -- start start 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9b0916640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 0x7fb9ac15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9b0916640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac161470 0x7fb9ac0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9b0916640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9b0916640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac06cf40 con 0x7fb9ac161850 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9b0916640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac06d370 con 0x7fb9ac161470 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9b0916640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0c8510 con 0x7fb9ac06bcb0 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.256+0000 7fb9aad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38276/0 (socket says 172.21.15.105:38276) 2024-03-08T08:30:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aad9d640 1 -- 172.21.15.105:0/923685801 learned_addr learned my addr 172.21.15.105:0/923685801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aa59c640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 0x7fb9ac15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aad9d640 1 -- 172.21.15.105:0/923685801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 msgr2=0x7fb9ac15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aad9d640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 0x7fb9ac15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9a9d9b640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac161470 0x7fb9ac0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aad9d640 1 -- 172.21.15.105:0/923685801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac161470 msgr2=0x7fb9ac0c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aad9d640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac161470 0x7fb9ac0c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aad9d640 1 -- 172.21.15.105:0/923685801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac0cd750 con 0x7fb9ac161850 2024-03-08T08:30:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9aad9d640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0c7960 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009d90 tx=0x7fb99c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9a2ffd640 1 -- 172.21.15.105:0/923685801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c039050 con 0x7fb9ac161850 2024-03-08T08:30:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9a2ffd640 1 -- 172.21.15.105:0/923685801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c045070 con 0x7fb9ac161850 2024-03-08T08:30:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac0cc990 con 0x7fb9ac161850 2024-03-08T08:30:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9a2ffd640 1 -- 172.21.15.105:0/923685801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c03e830 con 0x7fb9ac161850 2024-03-08T08:30:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.257+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac0ccf50 con 0x7fb9ac161850 2024-03-08T08:30:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.258+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac075120 con 0x7fb9ac161850 2024-03-08T08:30:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.258+0000 7fb9a2ffd640 1 -- 172.21.15.105:0/923685801 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb99c03e9d0 con 0x7fb9ac161850 2024-03-08T08:30:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.259+0000 7fb9a2ffd640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb98c06b500 0x7fb98c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.259+0000 7fb9a2ffd640 1 -- 172.21.15.105:0/923685801 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb99c0acd80 con 0x7fb9ac161850 2024-03-08T08:30:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.259+0000 7fb9aa59c640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb98c06b500 0x7fb98c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.259+0000 7fb9aa59c640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb98c06b500 0x7fb98c06d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4007fd0 tx=0x7fb9a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.263+0000 7fb9a2ffd640 1 -- 172.21.15.105:0/923685801 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb99c032e20 con 0x7fb9ac161850 2024-03-08T08:30:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.435+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9ac160900 con 0x7fb98c06b500 2024-03-08T08:30:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.435+0000 7fb9a2ffd640 1 -- 172.21.15.105:0/923685801 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb9ac160900 con 0x7fb98c06b500 2024-03-08T08:30:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 3 to deep-scrub 2024-03-08T08:30:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb98c06b500 msgr2=0x7fb98c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb98c06b500 0x7fb98c06d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4007fd0 tx=0x7fb9a4009040 comp rx=0 tx=0).stop 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 msgr2=0x7fb9ac0c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0c7960 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009d90 tx=0x7fb99c004a40 comp rx=0 tx=0).stop 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 shutdown_connections 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fb98c06b500 0x7fb98c06d9c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ac161850 0x7fb9ac0c7960 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9ac161470 0x7fb9ac0c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 --2- 172.21.15.105:0/923685801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9ac06bcb0 0x7fb9ac15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 >> 172.21.15.105:0/923685801 conn(0x7fb9ac0d3ce0 msgr2=0x7fb9ac0d40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 shutdown_connections 2024-03-08T08:30:04.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.437+0000 7fb9b0916640 1 -- 172.21.15.105:0/923685801 wait complete. 2024-03-08T08:30:04.449 INFO:tasks.ceph:Scrubbing osd.4 2024-03-08T08:30:04.449 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-08T08:30:04.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.552+0000 7f2beb2ed640 1 Processor -- start 2024-03-08T08:30:04.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.552+0000 7f2beb2ed640 1 -- start start 2024-03-08T08:30:04.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2beb2ed640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2be4144a50 0x7f2be40cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2beb2ed640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be406de60 0x7f2be40b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2beb2ed640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be40cf330 con 0x7f2be4144e30 2024-03-08T08:30:04.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2beb2ed640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be41458a0 con 0x7f2be4144a50 2024-03-08T08:30:04.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2beb2ed640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be406e610 con 0x7f2be406de60 2024-03-08T08:30:04.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2be9089640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2be4144a50 0x7f2be40cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2be9089640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2be4144a50 0x7f2be40cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38288/0 (socket says 172.21.15.105:38288) 2024-03-08T08:30:04.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2be9089640 1 -- 172.21.15.105:0/3629625968 learned_addr learned my addr 172.21.15.105:0/3629625968 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:04.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2be9089640 1 -- 172.21.15.105:0/3629625968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be406de60 msgr2=0x7f2be40b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2be9089640 1 --2- 172.21.15.105:0/3629625968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be406de60 0x7f2be40b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2be9089640 1 -- 172.21.15.105:0/3629625968 >> v1:172.21.15.144:6789/0 conn(0x7f2be4144e30 legacy=0x7f2be406d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.553+0000 7f2be9089640 1 -- 172.21.15.105:0/3629625968 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40cf5a0 con 0x7f2be4144a50 2024-03-08T08:30:04.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.554+0000 7f2be9089640 1 --2- 172.21.15.105:0/3629625968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144a50 0x7f2be40cc490 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009870 tx=0x7f2be002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfe9fffdb950 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.554+0000 7f2bdbfff640 1 -- 172.21.15.105:0/3629625968 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0035020 con 0x7f2be4144a50 2024-03-08T08:30:04.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.554+0000 7f2bdbfff640 1 -- 172.21.15.105:0/3629625968 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be0031640 con 0x7f2be4144a50 2024-03-08T08:30:04.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.554+0000 7f2bdbfff640 1 -- 172.21.15.105:0/3629625968 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be002d040 con 0x7f2be4144a50 2024-03-08T08:30:04.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.554+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3629625968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144a50 msgr2=0x7f2be40cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.554+0000 7f2beb2ed640 1 --2- 172.21.15.105:0/3629625968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144a50 0x7f2be40cc490 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009870 tx=0x7f2be002b8f0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.555+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3629625968 shutdown_connections 2024-03-08T08:30:04.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.555+0000 7f2beb2ed640 1 --2- 172.21.15.105:0/3629625968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be406de60 0x7f2be40b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.555+0000 7f2beb2ed640 1 --2- 172.21.15.105:0/3629625968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144a50 0x7f2be40cc490 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.555+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3629625968 >> 172.21.15.105:0/3629625968 conn(0x7f2be40b17b0 msgr2=0x7f2be40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.555+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3629625968 shutdown_connections 2024-03-08T08:30:04.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.555+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3629625968 wait complete. 2024-03-08T08:30:04.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.556+0000 7f2beb2ed640 1 Processor -- start 2024-03-08T08:30:04.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.556+0000 7f2beb2ed640 1 -- start start 2024-03-08T08:30:04.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.556+0000 7f2beb2ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be406de60 0x7f2be415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.556+0000 7f2beb2ed640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be4144a50 0x7f2be413dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.556+0000 7f2beb2ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144e30 0x7f2be41f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.556+0000 7f2beb2ed640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b3390 con 0x7f2be4144e30 2024-03-08T08:30:04.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.556+0000 7f2beb2ed640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be4145290 con 0x7f2be406de60 2024-03-08T08:30:04.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.556+0000 7f2beb2ed640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40b3510 con 0x7f2be4144a50 2024-03-08T08:30:04.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144e30 0x7f2be41f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144e30 0x7f2be41f7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38302/0 (socket says 172.21.15.105:38302) 2024-03-08T08:30:04.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 -- 172.21.15.105:0/3607260511 learned_addr learned my addr 172.21.15.105:0/3607260511 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 -- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be4144a50 msgr2=0x7f2be413dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be9089640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be406de60 0x7f2be415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2be4144a50 0x7f2be413dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 -- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be406de60 msgr2=0x7f2be415b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be406de60 0x7f2be415b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 -- 172.21.15.105:0/3607260511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be413e830 con 0x7f2be4144e30 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2be988a640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144e30 0x7f2be41f7250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00b970 tx=0x7f2bdc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc019070 con 0x7f2be4144e30 2024-03-08T08:30:04.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc00cb60 con 0x7f2be4144e30 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be415e470 con 0x7f2be4144e30 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc014440 con 0x7f2be4144e30 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.557+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be415ea30 con 0x7f2be4144e30 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.558+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2be4075200 con 0x7f2be4144e30 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bdc01c070 con 0x7f2be4144e30 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2bd9ffb640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2bc806b500 0x7f2bc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2bdc089c50 con 0x7f2be4144e30 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2be9089640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2bc806b500 0x7f2bc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2bd9ffb640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f2bc80713c0 0x7f2bc80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 --> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2bc8073e90 con 0x7f2bc80713c0 2024-03-08T08:30:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bdc08c350 con 0x7f2be4144e30 2024-03-08T08:30:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2be8888640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f2bc80713c0 0x7f2bc80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.559+0000 7f2be9089640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2bc806b500 0x7f2bc806d9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2be0036000 tx=0x7f2be003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.560+0000 7f2be8888640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f2bc80713c0 0x7f2bc80737e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.561+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 <== osd.4 v2:172.21.15.144:6801/3800656202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2bc8073e90 con 0x7f2bc80713c0 2024-03-08T08:30:04.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 --> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2be406e350 con 0x7f2bc80713c0 2024-03-08T08:30:04.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2bd9ffb640 1 -- 172.21.15.105:0/3607260511 <== osd.4 v2:172.21.15.144:6801/3800656202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2be406e350 con 0x7f2bc80713c0 2024-03-08T08:30:04.575 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:30:04.575 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T08:30:04.575 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:30:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f2bc80713c0 msgr2=0x7f2bc80737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2beb2ed640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.144:6801/3800656202,v1:172.21.15.144:6802/3800656202] conn(0x7f2bc80713c0 0x7f2bc80737e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2bc806b500 msgr2=0x7f2bc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2beb2ed640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2bc806b500 0x7f2bc806d9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2be0036000 tx=0x7f2be003c040 comp rx=0 tx=0).stop 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144e30 msgr2=0x7f2be41f7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2beb2ed640 1 --2- 172.21.15.105:0/3607260511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2be4144e30 0x7f2be41f7250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00b970 tx=0x7f2bdc00be30 comp rx=0 tx=0).stop 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.574+0000 7f2be988a640 1 -- 172.21.15.105:0/3607260511 reap_dead start 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.575+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 shutdown_connections 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.575+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 >> 172.21.15.105:0/3607260511 conn(0x7f2be40b17b0 msgr2=0x7f2be40b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.575+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 shutdown_connections 2024-03-08T08:30:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.575+0000 7f2beb2ed640 1 -- 172.21.15.105:0/3607260511 wait complete. 2024-03-08T08:30:04.586 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-08T08:30:04.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.685+0000 7fcc1fd77640 1 Processor -- start 2024-03-08T08:30:04.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.686+0000 7fcc1fd77640 1 -- start start 2024-03-08T08:30:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.686+0000 7fcc1fd77640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc18161470 0x7fcc1806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.686+0000 7fcc1fd77640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc180d48a0 con 0x7fcc1806bcb0 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.686+0000 7fcc1fd77640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc180d4b80 con 0x7fcc18161850 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.686+0000 7fcc1fd77640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1806c170 con 0x7fcc18161470 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.686+0000 7fcc1e314640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcc1806bcb0 0x7fcc180b2450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55810/0 (socket says 172.21.15.105:55810) 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.686+0000 7fcc1e314640 1 -- 172.21.15.105:0/3795874017 learned_addr learned my addr 172.21.15.105:0/3795874017 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1db13640 1 --2- 172.21.15.105:0/3795874017 >> v2:172.21.15.144:3300/0 conn(0x7fcc18161470 0x7fcc1806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590352487 0 0) 0x7fcc180d48a0 con 0x7fcc1806bcb0 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc04003520 con 0x7fcc1806bcb0 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4022582761 0 0) 0x7fcc04003520 con 0x7fcc1806bcb0 2024-03-08T08:30:04.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 >> v2:172.21.15.144:3300/0 conn(0x7fcc18161470 msgr2=0x7fcc1806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 --2- 172.21.15.105:0/3795874017 >> v2:172.21.15.144:3300/0 conn(0x7fcc18161470 0x7fcc1806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 >> v1:172.21.15.144:6789/0 conn(0x7fcc18161850 legacy=0x7fcc1806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1806c450 con 0x7fcc1806bcb0 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7fcc100019f0 con 0x7fcc1806bcb0 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 >> v1:172.21.15.105:6789/0 conn(0x7fcc1806bcb0 legacy=0x7fcc180b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc040039b0 0x7fcc04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc04004800 0x7fcc04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc040051c0 0x7fcc0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1806c170 con 0x7fcc040051c0 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0400c160 con 0x7fcc04004800 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0400c360 con 0x7fcc040039b0 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1e314640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc040051c0 0x7fcc0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1d312640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc04004800 0x7fcc04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1db13640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc040039b0 0x7fcc04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1e314640 1 -- 172.21.15.105:0/3795874017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc040039b0 msgr2=0x7fcc04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1e314640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc040039b0 0x7fcc04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1e314640 1 -- 172.21.15.105:0/3795874017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc04004800 msgr2=0x7fcc04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1e314640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc04004800 0x7fcc04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.687+0000 7fcc1e314640 1 -- 172.21.15.105:0/3795874017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04003520 con 0x7fcc040051c0 2024-03-08T08:30:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.688+0000 7fcc1e314640 1 -- 172.21.15.105:0/3795874017 reap_dead start 2024-03-08T08:30:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.688+0000 7fcc1e314640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc040051c0 0x7fcc0400b9c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000a860 tx=0x7fcc04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e351749c644ae90 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.688+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000a480 con 0x7fcc040051c0 2024-03-08T08:30:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.688+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10034630 con 0x7fcc040051c0 2024-03-08T08:30:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.688+0000 7fcc1cb11640 1 -- 172.21.15.105:0/3795874017 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10034a20 con 0x7fcc040051c0 2024-03-08T08:30:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.689+0000 7fcc1fd77640 1 -- 172.21.15.105:0/3795874017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc040051c0 msgr2=0x7fcc0400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.689+0000 7fcc1fd77640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc040051c0 0x7fcc0400b9c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000a860 tx=0x7fcc04005830 comp rx=0 tx=0).stop 2024-03-08T08:30:04.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.689+0000 7fcc1fd77640 1 -- 172.21.15.105:0/3795874017 shutdown_connections 2024-03-08T08:30:04.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.689+0000 7fcc1fd77640 1 --2- 172.21.15.105:0/3795874017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc040051c0 0x7fcc0400b9c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.689+0000 7fcc1fd77640 1 -- 172.21.15.105:0/3795874017 >> 172.21.15.105:0/3795874017 conn(0x7fcc180d3ce0 msgr2=0x7fcc180d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.689+0000 7fcc1fd77640 1 -- 172.21.15.105:0/3795874017 shutdown_connections 2024-03-08T08:30:04.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.689+0000 7fcc1fd77640 1 -- 172.21.15.105:0/3795874017 wait complete. 2024-03-08T08:30:04.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.690+0000 7fcc1fd77640 1 Processor -- start 2024-03-08T08:30:04.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.690+0000 7fcc1fd77640 1 -- start start 2024-03-08T08:30:04.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1fd77640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc040051c0 0x7fcc18158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1fd77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc180ca300 0x7fcc181f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1fd77640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc181f7410 0x7fcc181f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1fd77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1806d4f0 con 0x7fcc180ca300 2024-03-08T08:30:04.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1fd77640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1806d370 con 0x7fcc040051c0 2024-03-08T08:30:04.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1fd77640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1806d670 con 0x7fcc181f7410 2024-03-08T08:30:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc180ca300 0x7fcc181f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc180ca300 0x7fcc181f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38324/0 (socket says 172.21.15.105:38324) 2024-03-08T08:30:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 -- 172.21.15.105:0/2776610889 learned_addr learned my addr 172.21.15.105:0/2776610889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1e314640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc181f7410 0x7fcc181f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1db13640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc040051c0 0x7fcc18158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 -- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc181f7410 msgr2=0x7fcc181f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc181f7410 0x7fcc181f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 -- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc040051c0 msgr2=0x7fcc18158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc040051c0 0x7fcc18158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 -- 172.21.15.105:0/2776610889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc181f3790 con 0x7fcc180ca300 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1e314640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc181f7410 0x7fcc181f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1db13640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc040051c0 0x7fcc18158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc1d312640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc180ca300 0x7fcc181f6ed0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800f5c0 tx=0x7fcc0800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.691+0000 7fcc0e7fc640 1 -- 172.21.15.105:0/2776610889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc080052e0 con 0x7fcc180ca300 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.692+0000 7fcc0e7fc640 1 -- 172.21.15.105:0/2776610889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc08018070 con 0x7fcc180ca300 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.692+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc180c9aa0 con 0x7fcc180ca300 2024-03-08T08:30:04.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.692+0000 7fcc0e7fc640 1 -- 172.21.15.105:0/2776610889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08003830 con 0x7fcc180ca300 2024-03-08T08:30:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.692+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc180caf50 con 0x7fcc180ca300 2024-03-08T08:30:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.692+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1815b8f0 con 0x7fcc180ca300 2024-03-08T08:30:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.693+0000 7fcc0e7fc640 1 -- 172.21.15.105:0/2776610889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc080039d0 con 0x7fcc180ca300 2024-03-08T08:30:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.693+0000 7fcc0e7fc640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcbfc06b500 0x7fcbfc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.693+0000 7fcc0e7fc640 1 -- 172.21.15.105:0/2776610889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcc08089c00 con 0x7fcc180ca300 2024-03-08T08:30:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.693+0000 7fcc1db13640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcbfc06b500 0x7fcbfc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.694+0000 7fcc1db13640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcbfc06b500 0x7fcbfc06d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009a40 tx=0x7fcc14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.697+0000 7fcc0e7fc640 1 -- 172.21.15.105:0/2776610889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc08090080 con 0x7fcc180ca300 2024-03-08T08:30:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.867+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc180ca780 con 0x7fcbfc06b500 2024-03-08T08:30:04.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.868+0000 7fcc0e7fc640 1 -- 172.21.15.105:0/2776610889 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcc14013a20 con 0x7fcbfc06b500 2024-03-08T08:30:04.869 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 4 to deep-scrub 2024-03-08T08:30:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcbfc06b500 msgr2=0x7fcbfc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcbfc06b500 0x7fcbfc06d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009a40 tx=0x7fcc14008040 comp rx=0 tx=0).stop 2024-03-08T08:30:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc180ca300 msgr2=0x7fcc181f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc180ca300 0x7fcc181f6ed0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800f5c0 tx=0x7fcc0800fa80 comp rx=0 tx=0).stop 2024-03-08T08:30:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 shutdown_connections 2024-03-08T08:30:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fcbfc06b500 0x7fcbfc06d9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc181f7410 0x7fcc181f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc180ca300 0x7fcc181f6ed0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 --2- 172.21.15.105:0/2776610889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc040051c0 0x7fcc18158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 >> 172.21.15.105:0/2776610889 conn(0x7fcc180d3ce0 msgr2=0x7fcc180b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 shutdown_connections 2024-03-08T08:30:04.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.869+0000 7fcc1fd77640 1 -- 172.21.15.105:0/2776610889 wait complete. 2024-03-08T08:30:04.881 INFO:tasks.ceph:Scrubbing osd.5 2024-03-08T08:30:04.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-08T08:30:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.983+0000 7f51c53dd640 1 Processor -- start 2024-03-08T08:30:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.983+0000 7f51c53dd640 1 -- start start 2024-03-08T08:30:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51c53dd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f51c006da40 0x7f51c006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51c53dd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c0162c00 con 0x7f51c01624b0 2024-03-08T08:30:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51c53dd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c006fbf0 con 0x7f51c006f3a0 2024-03-08T08:30:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51c53dd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c006de20 con 0x7f51c006da40 2024-03-08T08:30:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51bf7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f51c01624b0 0x7f51c006bf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55812/0 (socket says 172.21.15.105:55812) 2024-03-08T08:30:04.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51bf7fe640 1 -- 172.21.15.105:0/4034279584 learned_addr learned my addr 172.21.15.105:0/4034279584 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810069944 0 0) 0x7f51b8001110 con 0x7f51c01624b0 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a8003520 con 0x7f51c01624b0 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.984+0000 7f51beffd640 1 --2- 172.21.15.105:0/4034279584 >> v2:172.21.15.144:3300/0 conn(0x7f51c006da40 0x7f51c006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285064401 0 0) 0x7f51a8003520 con 0x7f51c01624b0 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 >> v2:172.21.15.144:3300/0 conn(0x7f51c006da40 msgr2=0x7f51c006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 --2- 172.21.15.105:0/4034279584 >> v2:172.21.15.144:3300/0 conn(0x7f51c006da40 0x7f51c006ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 >> v1:172.21.15.144:6789/0 conn(0x7f51c006f3a0 legacy=0x7f51c0161da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c006f910 con 0x7f51c01624b0 2024-03-08T08:30:04.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f51b8001840 con 0x7f51c01624b0 2024-03-08T08:30:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 >> v1:172.21.15.105:6789/0 conn(0x7f51c01624b0 legacy=0x7f51c006bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80039b0 0x7f51a8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a8004800 0x7f51a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a80051c0 0x7f51a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c006de20 con 0x7f51a80051c0 2024-03-08T08:30:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a800c0f0 con 0x7f51a8004800 2024-03-08T08:30:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a800c2f0 con 0x7f51a80039b0 2024-03-08T08:30:04.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bf7fe640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a80051c0 0x7f51a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51be7fc640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a8004800 0x7f51a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51beffd640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80039b0 0x7f51a8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bf7fe640 1 -- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80039b0 msgr2=0x7f51a8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bf7fe640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80039b0 0x7f51a8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bf7fe640 1 -- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a8004800 msgr2=0x7f51a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:04.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bf7fe640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a8004800 0x7f51a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51bf7fe640 1 -- 172.21.15.105:0/4034279584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8003520 con 0x7f51a80051c0 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.985+0000 7f51beffd640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80039b0 0x7f51a8003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.986+0000 7f51bf7fe640 1 -- 172.21.15.105:0/4034279584 reap_dead start 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.986+0000 7f51bf7fe640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a80051c0 0x7f51a800b9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f51c0071870 tx=0x7f51a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cd98c4e29c047f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.986+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b800c0e0 con 0x7f51a80051c0 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.986+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b8030070 con 0x7f51a80051c0 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.986+0000 7f51bdffb640 1 -- 172.21.15.105:0/4034279584 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b80058d0 con 0x7f51a80051c0 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.987+0000 7f51c53dd640 1 -- 172.21.15.105:0/4034279584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a80051c0 msgr2=0x7f51a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.987+0000 7f51c53dd640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a80051c0 0x7f51a800b9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f51c0071870 tx=0x7f51a8005830 comp rx=0 tx=0).stop 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.987+0000 7f51c53dd640 1 -- 172.21.15.105:0/4034279584 shutdown_connections 2024-03-08T08:30:04.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.987+0000 7f51c53dd640 1 --2- 172.21.15.105:0/4034279584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a80051c0 0x7f51a800b9c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.987+0000 7f51c53dd640 1 -- 172.21.15.105:0/4034279584 >> 172.21.15.105:0/4034279584 conn(0x7f51c00b3590 msgr2=0x7f51c00b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.987+0000 7f51c53dd640 1 -- 172.21.15.105:0/4034279584 shutdown_connections 2024-03-08T08:30:04.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.987+0000 7f51c53dd640 1 -- 172.21.15.105:0/4034279584 wait complete. 2024-03-08T08:30:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.988+0000 7f51c53dd640 1 Processor -- start 2024-03-08T08:30:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.988+0000 7f51c53dd640 1 -- start start 2024-03-08T08:30:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51c53dd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80051c0 0x7f51c015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51c53dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51c01f4280 0x7f51c01f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51c53dd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c00b1f10 0x7f51c00b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51c53dd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00d0ca0 con 0x7f51c01f4280 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51c53dd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c006f200 con 0x7f51c00b1f10 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51c53dd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c006ecc0 con 0x7f51a80051c0 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51c01f4280 0x7f51c01f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51c01f4280 0x7f51c01f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38338/0 (socket says 172.21.15.105:38338) 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 -- 172.21.15.105:0/1985320125 learned_addr learned my addr 172.21.15.105:0/1985320125 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51beffd640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80051c0 0x7f51c015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51bf7fe640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c00b1f10 0x7f51c00b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 -- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80051c0 msgr2=0x7f51c015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a80051c0 0x7f51c015ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 -- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c00b1f10 msgr2=0x7f51c00b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c00b1f10 0x7f51c00b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 -- 172.21.15.105:0/1985320125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c01f7c90 con 0x7f51c01f4280 2024-03-08T08:30:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51bf7fe640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c00b1f10 0x7f51c00b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:30:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51be7fc640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51c01f4280 0x7f51c01f7750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f51b400f650 tx=0x7f51b400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:04.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4017020 con 0x7f51c01f4280 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b4003860 con 0x7f51c01f4280 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c009f470 con 0x7f51c01f4280 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4003c50 con 0x7f51c01f4280 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.989+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c01f3a80 con 0x7f51c01f4280 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.990+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f51c01f46e0 con 0x7f51c01f4280 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.991+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51b401a020 con 0x7f51c01f4280 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.991+0000 7f51a77fe640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51ac06b4d0 0x7f51ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.991+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f51b4089d20 con 0x7f51c01f4280 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.991+0000 7f51beffd640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51ac06b4d0 0x7f51ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:04.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.991+0000 7f51a77fe640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f51ac071390 0x7f51ac0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.991+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 --> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51b4019e50 con 0x7f51ac071390 2024-03-08T08:30:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.991+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51b408c420 con 0x7f51c01f4280 2024-03-08T08:30:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.992+0000 7f51bf7fe640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f51ac071390 0x7f51ac0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.992+0000 7f51beffd640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51ac06b4d0 0x7f51ac06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51b00043f0 tx=0x7f51b0003bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.992+0000 7f51bf7fe640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f51ac071390 0x7f51ac0737b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:04.993+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 <== osd.5 v2:172.21.15.144:6816/56982278 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f51b4019e50 con 0x7f51ac071390 2024-03-08T08:30:05.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.006+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 --> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f51c01f4040 con 0x7f51ac071390 2024-03-08T08:30:05.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51a77fe640 1 -- 172.21.15.105:0/1985320125 <== osd.5 v2:172.21.15.144:6816/56982278 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f51c01f4040 con 0x7f51ac071390 2024-03-08T08:30:05.008 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:30:05.008 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T08:30:05.008 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:30:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f51ac071390 msgr2=0x7f51ac0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.144:6816/56982278,v1:172.21.15.144:6817/56982278] conn(0x7f51ac071390 0x7f51ac0737b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51ac06b4d0 msgr2=0x7f51ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f51ac06b4d0 0x7f51ac06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51b00043f0 tx=0x7f51b0003bf0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51c01f4280 msgr2=0x7f51c01f7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 --2- 172.21.15.105:0/1985320125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51c01f4280 0x7f51c01f7750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f51b400f650 tx=0x7f51b400fb10 comp rx=0 tx=0).stop 2024-03-08T08:30:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51bf7fe640 1 -- 172.21.15.105:0/1985320125 reap_dead start 2024-03-08T08:30:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 shutdown_connections 2024-03-08T08:30:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 >> 172.21.15.105:0/1985320125 conn(0x7f51c00b3590 msgr2=0x7f51c00ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 shutdown_connections 2024-03-08T08:30:05.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.007+0000 7f51c53dd640 1 -- 172.21.15.105:0/1985320125 wait complete. 2024-03-08T08:30:05.019 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-08T08:30:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.118+0000 7fdb213b6640 1 Processor -- start 2024-03-08T08:30:05.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.118+0000 7fdb213b6640 1 -- start start 2024-03-08T08:30:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb213b6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdb1c144a00 0x7fdb1c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb213b6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb1c06b390 0x7fdb1c0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb213b6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb1c1457b0 con 0x7fdb1c144f60 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb213b6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1c1454d0 con 0x7fdb1c144a00 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb213b6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c15c2c0 con 0x7fdb1c06b390 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb1affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdb1c144a00 0x7fdb1c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb1affd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdb1c144a00 0x7fdb1c15cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38350/0 (socket says 172.21.15.105:38350) 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb1affd640 1 -- 172.21.15.105:0/3533611365 learned_addr learned my addr 172.21.15.105:0/3533611365 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb1affd640 1 -- 172.21.15.105:0/3533611365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb1c06b390 msgr2=0x7fdb1c0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb1affd640 1 --2- 172.21.15.105:0/3533611365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb1c06b390 0x7fdb1c0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb1affd640 1 -- 172.21.15.105:0/3533611365 >> v1:172.21.15.144:6789/0 conn(0x7fdb1c144f60 legacy=0x7fdb1c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:05.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.119+0000 7fdb1affd640 1 -- 172.21.15.105:0/3533611365 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c15c5a0 con 0x7fdb1c144a00 2024-03-08T08:30:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb1affd640 1 --2- 172.21.15.105:0/3533611365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c15cda0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009e50 tx=0x7fdb0c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbba2559c4f4f03 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb19ffb640 1 -- 172.21.15.105:0/3533611365 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c02ca70 con 0x7fdb1c144a00 2024-03-08T08:30:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb19ffb640 1 -- 172.21.15.105:0/3533611365 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c02cc10 con 0x7fdb1c144a00 2024-03-08T08:30:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb19ffb640 1 -- 172.21.15.105:0/3533611365 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0343f0 con 0x7fdb1c144a00 2024-03-08T08:30:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb213b6640 1 -- 172.21.15.105:0/3533611365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 msgr2=0x7fdb1c15cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb213b6640 1 --2- 172.21.15.105:0/3533611365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c15cda0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009e50 tx=0x7fdb0c02bed0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb213b6640 1 -- 172.21.15.105:0/3533611365 shutdown_connections 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb213b6640 1 --2- 172.21.15.105:0/3533611365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb1c06b390 0x7fdb1c0cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.120+0000 7fdb213b6640 1 --2- 172.21.15.105:0/3533611365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c15cda0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.121+0000 7fdb213b6640 1 -- 172.21.15.105:0/3533611365 >> 172.21.15.105:0/3533611365 conn(0x7fdb1c06bd90 msgr2=0x7fdb1c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.121+0000 7fdb213b6640 1 -- 172.21.15.105:0/3533611365 shutdown_connections 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.121+0000 7fdb213b6640 1 -- 172.21.15.105:0/3533611365 wait complete. 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.122+0000 7fdb213b6640 1 Processor -- start 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.122+0000 7fdb213b6640 1 -- start start 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.122+0000 7fdb213b6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb1c06b390 0x7fdb1c15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb213b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c0cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb213b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c144f60 0x7fdb1c13e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb213b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c06ff10 con 0x7fdb1c144a00 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb213b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c15d2e0 con 0x7fdb1c144f60 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb213b6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c06c8b0 con 0x7fdb1c06b390 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c0cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c0cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38364/0 (socket says 172.21.15.105:38364) 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 -- 172.21.15.105:0/1398436230 learned_addr learned my addr 172.21.15.105:0/1398436230 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 -- 172.21.15.105:0/1398436230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb1c06b390 msgr2=0x7fdb1c15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb1c06b390 0x7fdb1c15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 -- 172.21.15.105:0/1398436230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c144f60 msgr2=0x7fdb1c13e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c144f60 0x7fdb1c13e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 -- 172.21.15.105:0/1398436230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c1e57b0 con 0x7fdb1c144a00 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb1a7fc640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c0cc060 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000b3e0 tx=0x7fdb1000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb037fe640 1 -- 172.21.15.105:0/1398436230 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10015020 con 0x7fdb1c144a00 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb037fe640 1 -- 172.21.15.105:0/1398436230 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb10011850 con 0x7fdb1c144a00 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1c1e5a90 con 0x7fdb1c144a00 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb037fe640 1 -- 172.21.15.105:0/1398436230 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10011c40 con 0x7fdb1c144a00 2024-03-08T08:30:05.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.123+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1c1e6050 con 0x7fdb1c144a00 2024-03-08T08:30:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.124+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1c1e6540 con 0x7fdb1c144a00 2024-03-08T08:30:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.125+0000 7fdb037fe640 1 -- 172.21.15.105:0/1398436230 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb10018020 con 0x7fdb1c144a00 2024-03-08T08:30:05.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.125+0000 7fdb037fe640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdb0806b4d0 0x7fdb0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.125+0000 7fdb037fe640 1 -- 172.21.15.105:0/1398436230 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdb10089de0 con 0x7fdb1c144a00 2024-03-08T08:30:05.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.125+0000 7fdb1affd640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdb0806b4d0 0x7fdb0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.126+0000 7fdb1affd640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdb0806b4d0 0x7fdb0806d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c035000 tx=0x7fdb0c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.130+0000 7fdb037fe640 1 -- 172.21.15.105:0/1398436230 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb1001f020 con 0x7fdb1c144a00 2024-03-08T08:30:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.302+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb1c075120 con 0x7fdb0806b4d0 2024-03-08T08:30:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.303+0000 7fdb037fe640 1 -- 172.21.15.105:0/1398436230 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdb1c075120 con 0x7fdb0806b4d0 2024-03-08T08:30:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 5 to deep-scrub 2024-03-08T08:30:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdb0806b4d0 msgr2=0x7fdb0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdb0806b4d0 0x7fdb0806d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c035000 tx=0x7fdb0c02e000 comp rx=0 tx=0).stop 2024-03-08T08:30:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 msgr2=0x7fdb1c0cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c0cc060 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000b3e0 tx=0x7fdb1000b8a0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 shutdown_connections 2024-03-08T08:30:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fdb0806b4d0 0x7fdb0806d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c144f60 0x7fdb1c13e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb1c144a00 0x7fdb1c0cc060 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 --2- 172.21.15.105:0/1398436230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb1c06b390 0x7fdb1c15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 >> 172.21.15.105:0/1398436230 conn(0x7fdb1c06bd90 msgr2=0x7fdb1c0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 shutdown_connections 2024-03-08T08:30:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.305+0000 7fdb213b6640 1 -- 172.21.15.105:0/1398436230 wait complete. 2024-03-08T08:30:05.316 INFO:tasks.ceph:Scrubbing osd.6 2024-03-08T08:30:05.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-08T08:30:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.419+0000 7f5891491640 1 Processor -- start 2024-03-08T08:30:05.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.419+0000 7f5891491640 1 -- start start 2024-03-08T08:30:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f5891491640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c15e110 0x7f588c0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f5891491640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f588c15e4f0 0x7f588c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f5891491640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f588c0d2140 0x7f588c0d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f5891491640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c0b0b00 con 0x7f588c0d2140 2024-03-08T08:30:05.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f5891491640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c0cf4a0 con 0x7f588c15e4f0 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f5891491640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0cf740 con 0x7f588c15e110 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f588c0d2140 0x7f588c0d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f588c0d2140 0x7f588c0d25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38380/0 (socket says 172.21.15.105:38380) 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588b7fe640 1 -- 172.21.15.105:0/4057761561 learned_addr learned my addr 172.21.15.105:0/4057761561 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588b7fe640 1 -- 172.21.15.105:0/4057761561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c15e110 msgr2=0x7f588c0d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588affd640 1 --2- 172.21.15.105:0/4057761561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c15e110 0x7f588c0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588a7fc640 1 --2- 172.21.15.105:0/4057761561 >> v2:172.21.15.144:3300/0 conn(0x7f588c15e4f0 0x7f588c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588b7fe640 1 --2- 172.21.15.105:0/4057761561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c15e110 0x7f588c0d4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588b7fe640 1 -- 172.21.15.105:0/4057761561 >> v2:172.21.15.144:3300/0 conn(0x7f588c15e4f0 msgr2=0x7f588c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588b7fe640 1 --2- 172.21.15.105:0/4057761561 >> v2:172.21.15.144:3300/0 conn(0x7f588c15e4f0 0x7f588c0d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.420+0000 7f588b7fe640 1 -- 172.21.15.105:0/4057761561 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c0cfa20 con 0x7f588c0d2140 2024-03-08T08:30:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.421+0000 7f588b7fe640 1 --2- 172.21.15.105:0/4057761561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c0d2140 0x7f588c0d25b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f588c08c6e0 tx=0x7f588402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83567aedb7e9636a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.421+0000 7f5889ffb640 1 -- 172.21.15.105:0/4057761561 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588402e070 con 0x7f588c0d2140 2024-03-08T08:30:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.421+0000 7f5889ffb640 1 -- 172.21.15.105:0/4057761561 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5884037070 con 0x7f588c0d2140 2024-03-08T08:30:05.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.421+0000 7f5889ffb640 1 -- 172.21.15.105:0/4057761561 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884031a00 con 0x7f588c0d2140 2024-03-08T08:30:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 -- 172.21.15.105:0/4057761561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c0d2140 msgr2=0x7f588c0d25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 --2- 172.21.15.105:0/4057761561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c0d2140 0x7f588c0d25b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f588c08c6e0 tx=0x7f588402b880 comp rx=0 tx=0).stop 2024-03-08T08:30:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 -- 172.21.15.105:0/4057761561 shutdown_connections 2024-03-08T08:30:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 --2- 172.21.15.105:0/4057761561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c0d2140 0x7f588c0d25b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 --2- 172.21.15.105:0/4057761561 >> v2:172.21.15.144:3300/0 conn(0x7f588c15e4f0 0x7f588c0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 --2- 172.21.15.105:0/4057761561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c15e110 0x7f588c0d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 -- 172.21.15.105:0/4057761561 >> 172.21.15.105:0/4057761561 conn(0x7f588c0d3d00 msgr2=0x7f588c0d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 -- 172.21.15.105:0/4057761561 shutdown_connections 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.422+0000 7f5891491640 1 -- 172.21.15.105:0/4057761561 wait complete. 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.423+0000 7f5891491640 1 Processor -- start 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.423+0000 7f5891491640 1 -- start start 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f5891491640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0d2140 0x7f588c1f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f5891491640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c15e110 0x7f588c153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f5891491640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c15e4f0 0x7f588c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f5891491640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0d0890 con 0x7f588c15e4f0 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f5891491640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0d0710 con 0x7f588c15e110 2024-03-08T08:30:05.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f5891491640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c0d0a10 con 0x7f588c0d2140 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c15e4f0 0x7f588c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c15e4f0 0x7f588c159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38392/0 (socket says 172.21.15.105:38392) 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 -- 172.21.15.105:0/2960021526 learned_addr learned my addr 172.21.15.105:0/2960021526 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588a7fc640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c15e110 0x7f588c153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588affd640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0d2140 0x7f588c1f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 -- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0d2140 msgr2=0x7f588c1f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0d2140 0x7f588c1f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 -- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c15e110 msgr2=0x7f588c153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c15e110 0x7f588c153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 -- 172.21.15.105:0/2960021526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c0cc040 con 0x7f588c15e4f0 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.424+0000 7f588b7fe640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c15e4f0 0x7f588c159150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f58840090b0 tx=0x7f58840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.425+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884032880 con 0x7f588c15e4f0 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.425+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5884032a20 con 0x7f588c15e4f0 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.425+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884032e10 con 0x7f588c15e4f0 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.425+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c0cc320 con 0x7f588c15e4f0 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.425+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c0cc8e0 con 0x7f588c15e4f0 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.426+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f588c0685c0 con 0x7f588c15e4f0 2024-03-08T08:30:05.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.426+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f588404c060 con 0x7f588c15e4f0 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.426+0000 7f58737fe640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f587806b4d0 0x7f587806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.427+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f58840ab970 con 0x7f588c15e4f0 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.427+0000 7f588affd640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f587806b4d0 0x7f587806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.427+0000 7f58737fe640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7f5878071390 0x7f58780737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.427+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 --> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5878073e60 con 0x7f5878071390 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.427+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58840b02c0 con 0x7f588c15e4f0 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.427+0000 7f588a7fc640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7f5878071390 0x7f58780737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.427+0000 7f588affd640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f587806b4d0 0x7f587806d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f587c007130 tx=0x7f587c007720 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.427+0000 7f588a7fc640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7f5878071390 0x7f58780737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.428+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 <== osd.6 v2:172.21.15.144:6800/768956787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5878073e60 con 0x7f5878071390 2024-03-08T08:30:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.441+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 --> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f588c075200 con 0x7f5878071390 2024-03-08T08:30:05.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f58737fe640 1 -- 172.21.15.105:0/2960021526 <== osd.6 v2:172.21.15.144:6800/768956787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f588c075200 con 0x7f5878071390 2024-03-08T08:30:05.443 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:30:05.443 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T08:30:05.443 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:30:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7f5878071390 msgr2=0x7f58780737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.144:6800/768956787,v1:172.21.15.144:6803/768956787] conn(0x7f5878071390 0x7f58780737b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f587806b4d0 msgr2=0x7f587806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f587806b4d0 0x7f587806d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f587c007130 tx=0x7f587c007720 comp rx=0 tx=0).stop 2024-03-08T08:30:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c15e4f0 msgr2=0x7f588c159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 --2- 172.21.15.105:0/2960021526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f588c15e4f0 0x7f588c159150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f58840090b0 tx=0x7f58840046c0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f588b7fe640 1 -- 172.21.15.105:0/2960021526 reap_dead start 2024-03-08T08:30:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 shutdown_connections 2024-03-08T08:30:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 >> 172.21.15.105:0/2960021526 conn(0x7f588c0d3d00 msgr2=0x7f588c0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 shutdown_connections 2024-03-08T08:30:05.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.442+0000 7f5891491640 1 -- 172.21.15.105:0/2960021526 wait complete. 2024-03-08T08:30:05.453 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-08T08:30:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.552+0000 7f65c37c9640 1 Processor -- start 2024-03-08T08:30:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.552+0000 7f65c37c9640 1 -- start start 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c37c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 0x7f65bc071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c37c9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65bc06e0a0 0x7f65bc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c37c9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65bc15b670 con 0x7f65bc0b4d00 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c37c9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc06ea50 con 0x7f65bc06e0a0 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c37c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc15b940 con 0x7f65bc072440 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c0d64640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f65bc0b4d00 0x7f65bc15af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55814/0 (socket says 172.21.15.105:55814) 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c0d64640 1 -- 172.21.15.105:0/1111308711 learned_addr learned my addr 172.21.15.105:0/1111308711 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c1d66640 1 --2- 172.21.15.105:0/1111308711 >> v2:172.21.15.144:3300/0 conn(0x7f65bc06e0a0 0x7f65bc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.553+0000 7f65c1565640 1 --2- 172.21.15.105:0/1111308711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 0x7f65bc071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.554+0000 7f65b3fff640 1 -- 172.21.15.105:0/1111308711 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067721186 0 0) 0x7f65bc15b670 con 0x7f65bc0b4d00 2024-03-08T08:30:05.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.554+0000 7f65b3fff640 1 -- 172.21.15.105:0/1111308711 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65a8003520 con 0x7f65bc0b4d00 2024-03-08T08:30:05.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.554+0000 7f65c1d66640 1 -- 172.21.15.105:0/1111308711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 msgr2=0x7f65bc071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:05.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.554+0000 7f65c1d66640 1 --2- 172.21.15.105:0/1111308711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 0x7f65bc071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.554+0000 7f65b3fff640 1 -- 172.21.15.105:0/1111308711 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2342701788 0 0) 0x7f65a8003520 con 0x7f65bc0b4d00 2024-03-08T08:30:05.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.554+0000 7f65c1d66640 1 -- 172.21.15.105:0/1111308711 >> v1:172.21.15.105:6789/0 conn(0x7f65bc0b4d00 legacy=0x7f65bc15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.554+0000 7f65c1d66640 1 -- 172.21.15.105:0/1111308711 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc15bc20 con 0x7f65bc06e0a0 2024-03-08T08:30:05.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65c1d66640 1 --2- 172.21.15.105:0/1111308711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65bc06e0a0 0x7f65bc06e510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65b40097a0 tx=0x7f65b402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=451372e4eb6c360 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65b3fff640 1 -- 172.21.15.105:0/1111308711 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b4037070 con 0x7f65bc06e0a0 2024-03-08T08:30:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65b3fff640 1 -- 172.21.15.105:0/1111308711 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b4030510 con 0x7f65bc06e0a0 2024-03-08T08:30:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65b3fff640 1 -- 172.21.15.105:0/1111308711 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac001670 con 0x7f65bc06e0a0 2024-03-08T08:30:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65c37c9640 1 -- 172.21.15.105:0/1111308711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65bc06e0a0 msgr2=0x7f65bc06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1111308711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65bc06e0a0 0x7f65bc06e510 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65b40097a0 tx=0x7f65b402b790 comp rx=0 tx=0).stop 2024-03-08T08:30:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65c37c9640 1 -- 172.21.15.105:0/1111308711 shutdown_connections 2024-03-08T08:30:05.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1111308711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65bc06e0a0 0x7f65bc06e510 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1111308711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 0x7f65bc071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.555+0000 7f65c37c9640 1 -- 172.21.15.105:0/1111308711 >> 172.21.15.105:0/1111308711 conn(0x7f65bc0b1740 msgr2=0x7f65bc0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.556+0000 7f65c37c9640 1 -- 172.21.15.105:0/1111308711 shutdown_connections 2024-03-08T08:30:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.556+0000 7f65c37c9640 1 -- 172.21.15.105:0/1111308711 wait complete. 2024-03-08T08:30:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c37c9640 1 Processor -- start 2024-03-08T08:30:05.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c37c9640 1 -- start start 2024-03-08T08:30:05.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c37c9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc06e0a0 0x7f65bc1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c37c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 0x7f65bc14ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c37c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65bc0b4d00 0x7f65bc146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c37c9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc071560 con 0x7f65bc06e0a0 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c37c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc070b20 con 0x7f65bc0b4d00 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c37c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc072820 con 0x7f65bc072440 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c1565640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc06e0a0 0x7f65bc1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c1565640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc06e0a0 0x7f65bc1596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38408/0 (socket says 172.21.15.105:38408) 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.557+0000 7f65c1565640 1 -- 172.21.15.105:0/1740391354 learned_addr learned my addr 172.21.15.105:0/1740391354 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:05.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65c1565640 1 -- 172.21.15.105:0/1740391354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 msgr2=0x7f65bc14ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65c1565640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 0x7f65bc14ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65c1565640 1 -- 172.21.15.105:0/1740391354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65bc0b4d00 msgr2=0x7f65bc146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65c1565640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65bc0b4d00 0x7f65bc146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65c1565640 1 -- 172.21.15.105:0/1740391354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0c92c0 con 0x7f65bc06e0a0 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65c1565640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc06e0a0 0x7f65bc1596e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f65b800efc0 tx=0x7f65b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65b1ffb640 1 -- 172.21.15.105:0/1740391354 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8018020 con 0x7f65bc06e0a0 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65b1ffb640 1 -- 172.21.15.105:0/1740391354 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b8014480 con 0x7f65bc06e0a0 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc0c95a0 con 0x7f65bc06e0a0 2024-03-08T08:30:05.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65b1ffb640 1 -- 172.21.15.105:0/1740391354 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8014850 con 0x7f65bc06e0a0 2024-03-08T08:30:05.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.558+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc0c9b60 con 0x7f65bc06e0a0 2024-03-08T08:30:05.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.559+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc075120 con 0x7f65bc06e0a0 2024-03-08T08:30:05.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.560+0000 7f65b1ffb640 1 -- 172.21.15.105:0/1740391354 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65b8005d10 con 0x7f65bc06e0a0 2024-03-08T08:30:05.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.560+0000 7f65b1ffb640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f65a006b500 0x7f65a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.560+0000 7f65c0d64640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f65a006b500 0x7f65a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.560+0000 7f65b1ffb640 1 -- 172.21.15.105:0/1740391354 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65b808aa30 con 0x7f65bc06e0a0 2024-03-08T08:30:05.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.561+0000 7f65c0d64640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f65a006b500 0x7f65a006d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f65ac001eb0 tx=0x7f65ac001a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.564+0000 7f65b1ffb640 1 -- 172.21.15.105:0/1740391354 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65b8022070 con 0x7f65bc06e0a0 2024-03-08T08:30:05.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.734+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f65bc1518b0 con 0x7f65a006b500 2024-03-08T08:30:05.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.735+0000 7f65b1ffb640 1 -- 172.21.15.105:0/1740391354 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f65bc1518b0 con 0x7f65a006b500 2024-03-08T08:30:05.736 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 6 to deep-scrub 2024-03-08T08:30:05.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f65a006b500 msgr2=0x7f65a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f65a006b500 0x7f65a006d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f65ac001eb0 tx=0x7f65ac001a40 comp rx=0 tx=0).stop 2024-03-08T08:30:05.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc06e0a0 msgr2=0x7f65bc1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc06e0a0 0x7f65bc1596e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f65b800efc0 tx=0x7f65b800c490 comp rx=0 tx=0).stop 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 shutdown_connections 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f65a006b500 0x7f65a006d9c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65bc0b4d00 0x7f65bc146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65bc072440 0x7f65bc14ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 --2- 172.21.15.105:0/1740391354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65bc06e0a0 0x7f65bc1596e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 >> 172.21.15.105:0/1740391354 conn(0x7f65bc0b1740 msgr2=0x7f65bc15e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 shutdown_connections 2024-03-08T08:30:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.737+0000 7f65c37c9640 1 -- 172.21.15.105:0/1740391354 wait complete. 2024-03-08T08:30:05.749 INFO:tasks.ceph:Scrubbing osd.7 2024-03-08T08:30:05.749 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-08T08:30:05.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4aa1215640 1 Processor -- start 2024-03-08T08:30:05.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4aa1215640 1 -- start start 2024-03-08T08:30:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4aa1215640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a9c15e110 0x7f4a9c0d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4aa1215640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4a9c15e4f0 0x7f4a9c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4aa1215640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 0x7f4a9c0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4aa1215640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9c0b0b00 con 0x7f4a9c15e110 2024-03-08T08:30:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4aa1215640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a9c0cf4a0 con 0x7f4a9c15e4f0 2024-03-08T08:30:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4aa1215640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0cf740 con 0x7f4a9c0d2140 2024-03-08T08:30:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4a9ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a9c15e110 0x7f4a9c0d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4a9ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4a9c15e110 0x7f4a9c0d4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38418/0 (socket says 172.21.15.105:38418) 2024-03-08T08:30:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.852+0000 7f4a9ad9d640 1 -- 172.21.15.105:0/2126170357 learned_addr learned my addr 172.21.15.105:0/2126170357 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a93fff640 1 --2- 172.21.15.105:0/2126170357 >> v2:172.21.15.144:3300/0 conn(0x7f4a9c15e4f0 0x7f4a9c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a9b59e640 1 --2- 172.21.15.105:0/2126170357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 0x7f4a9c0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a9ad9d640 1 -- 172.21.15.105:0/2126170357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 msgr2=0x7f4a9c0d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a9ad9d640 1 --2- 172.21.15.105:0/2126170357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 0x7f4a9c0d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a9ad9d640 1 -- 172.21.15.105:0/2126170357 >> v2:172.21.15.144:3300/0 conn(0x7f4a9c15e4f0 msgr2=0x7f4a9c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a9ad9d640 1 --2- 172.21.15.105:0/2126170357 >> v2:172.21.15.144:3300/0 conn(0x7f4a9c15e4f0 0x7f4a9c0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a9ad9d640 1 -- 172.21.15.105:0/2126170357 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c0cfa20 con 0x7f4a9c15e110 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a93fff640 1 --2- 172.21.15.105:0/2126170357 >> v2:172.21.15.144:3300/0 conn(0x7f4a9c15e4f0 0x7f4a9c0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a9b59e640 1 --2- 172.21.15.105:0/2126170357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 0x7f4a9c0d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.853+0000 7f4a9ad9d640 1 --2- 172.21.15.105:0/2126170357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 0x7f4a9c0d4820 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c08c710 tx=0x7f4a8c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224cd42abac7ba44 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4a9a59c640 1 -- 172.21.15.105:0/2126170357 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0317d0 con 0x7f4a9c15e110 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4a9a59c640 1 -- 172.21.15.105:0/2126170357 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c037070 con 0x7f4a9c15e110 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4a9a59c640 1 -- 172.21.15.105:0/2126170357 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c031b90 con 0x7f4a9c15e110 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 -- 172.21.15.105:0/2126170357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 msgr2=0x7f4a9c0d4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 --2- 172.21.15.105:0/2126170357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 0x7f4a9c0d4820 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c08c710 tx=0x7f4a8c02b870 comp rx=0 tx=0).stop 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 -- 172.21.15.105:0/2126170357 shutdown_connections 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 --2- 172.21.15.105:0/2126170357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 0x7f4a9c0d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 --2- 172.21.15.105:0/2126170357 >> v2:172.21.15.144:3300/0 conn(0x7f4a9c15e4f0 0x7f4a9c0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 --2- 172.21.15.105:0/2126170357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 0x7f4a9c0d4820 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 -- 172.21.15.105:0/2126170357 >> 172.21.15.105:0/2126170357 conn(0x7f4a9c0d3d00 msgr2=0x7f4a9c0d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 -- 172.21.15.105:0/2126170357 shutdown_connections 2024-03-08T08:30:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.854+0000 7f4aa1215640 1 -- 172.21.15.105:0/2126170357 wait complete. 2024-03-08T08:30:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.855+0000 7f4aa1215640 1 Processor -- start 2024-03-08T08:30:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.855+0000 7f4aa1215640 1 -- start start 2024-03-08T08:30:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4aa1215640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 0x7f4a9c15fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4aa1215640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 0x7f4a9c0b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4aa1215640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a9c15e4f0 0x7f4a9c147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4aa1215640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0d0890 con 0x7f4a9c15e110 2024-03-08T08:30:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4aa1215640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0d0710 con 0x7f4a9c15e4f0 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4aa1215640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0d0a10 con 0x7f4a9c0d2140 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 0x7f4a9c0b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 0x7f4a9c0b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38430/0 (socket says 172.21.15.105:38430) 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4a93fff640 1 -- 172.21.15.105:0/1368301262 learned_addr learned my addr 172.21.15.105:0/1368301262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4a9ad9d640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 0x7f4a9c15fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.856+0000 7f4a9b59e640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a9c15e4f0 0x7f4a9c147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a93fff640 1 -- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 msgr2=0x7f4a9c15fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a93fff640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a9c0d2140 0x7f4a9c15fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a93fff640 1 -- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a9c15e4f0 msgr2=0x7f4a9c147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a93fff640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a9c15e4f0 0x7f4a9c147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a93fff640 1 -- 172.21.15.105:0/1368301262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c15ff60 con 0x7f4a9c15e110 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a9b59e640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a9c15e4f0 0x7f4a9c147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a93fff640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 0x7f4a9c0b6b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c7f0 tx=0x7f4a8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84015070 con 0x7f4a9c15e110 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a84007820 con 0x7f4a9c15e110 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84007bd0 con 0x7f4a9c15e110 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c1601e0 con 0x7f4a9c15e110 2024-03-08T08:30:05.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.857+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c160a80 con 0x7f4a9c15e110 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.858+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a9c0d1140 con 0x7f4a9c15e110 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.858+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a84018070 con 0x7f4a9c15e110 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.859+0000 7f4a937fe640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4a7c06b500 0x7f4a7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.859+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4a84089f00 con 0x7f4a9c15e110 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.859+0000 7f4a9ad9d640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4a7c06b500 0x7f4a7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.859+0000 7f4a937fe640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7f4a7c0713c0 0x7f4a7c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.859+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 --> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a8401be50 con 0x7f4a7c0713c0 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.859+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a8408c600 con 0x7f4a9c15e110 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.859+0000 7f4a9ad9d640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4a7c06b500 0x7f4a7c06d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c036000 tx=0x7f4a8c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.859+0000 7f4a9b59e640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7f4a7c0713c0 0x7f4a7c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.860+0000 7f4a9b59e640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7f4a7c0713c0 0x7f4a7c0737e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.860+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 <== osd.7 v2:172.21.15.144:6824/1634585826 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4a8401be50 con 0x7f4a7c0713c0 2024-03-08T08:30:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.873+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 --> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4a9c075200 con 0x7f4a7c0713c0 2024-03-08T08:30:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4a937fe640 1 -- 172.21.15.105:0/1368301262 <== osd.7 v2:172.21.15.144:6824/1634585826 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4a9c075200 con 0x7f4a7c0713c0 2024-03-08T08:30:05.875 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-08T08:30:05.875 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-08T08:30:05.875 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-08T08:30:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7f4a7c0713c0 msgr2=0x7f4a7c0737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.144:6824/1634585826,v1:172.21.15.144:6825/1634585826] conn(0x7f4a7c0713c0 0x7f4a7c0737e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4a7c06b500 msgr2=0x7f4a7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4a7c06b500 0x7f4a7c06d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c036000 tx=0x7f4a8c03d040 comp rx=0 tx=0).stop 2024-03-08T08:30:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 msgr2=0x7f4a9c0b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 --2- 172.21.15.105:0/1368301262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a9c15e110 0x7f4a9c0b6b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400c7f0 tx=0x7f4a8400ccb0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4a9b59e640 1 -- 172.21.15.105:0/1368301262 reap_dead start 2024-03-08T08:30:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 shutdown_connections 2024-03-08T08:30:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 >> 172.21.15.105:0/1368301262 conn(0x7f4a9c0d3d00 msgr2=0x7f4a9c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 shutdown_connections 2024-03-08T08:30:05.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.874+0000 7f4aa1215640 1 -- 172.21.15.105:0/1368301262 wait complete. 2024-03-08T08:30:05.885 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-08T08:30:05.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.985+0000 7fd41e484640 1 Processor -- start 2024-03-08T08:30:05.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd41e484640 1 -- start start 2024-03-08T08:30:05.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd41e484640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 0x7fd41806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd41e484640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd418161850 0x7fd4180d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd41e484640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4180d48a0 con 0x7fd41806d370 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd41e484640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4180d2ba0 con 0x7fd418161850 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd41e484640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180d4b80 con 0x7fd418161470 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd4177fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd418161850 0x7fd4180d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd41ca21640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd41806d370 0x7fd41806b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55824/0 (socket says 172.21.15.105:55824) 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.986+0000 7fd41ca21640 1 -- 172.21.15.105:0/1321270565 learned_addr learned my addr 172.21.15.105:0/1321270565 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.987+0000 7fd417fff640 1 --2- 172.21.15.105:0/1321270565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 0x7fd41806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.987+0000 7fd4177fe640 1 -- 172.21.15.105:0/1321270565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 msgr2=0x7fd41806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.987+0000 7fd4177fe640 1 --2- 172.21.15.105:0/1321270565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 0x7fd41806c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.987+0000 7fd4177fe640 1 -- 172.21.15.105:0/1321270565 >> v1:172.21.15.105:6789/0 conn(0x7fd41806d370 legacy=0x7fd41806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.987+0000 7fd4177fe640 1 -- 172.21.15.105:0/1321270565 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41806c170 con 0x7fd418161850 2024-03-08T08:30:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.987+0000 7fd416ffd640 1 -- 172.21.15.105:0/1321270565 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098447325 0 0) 0x7fd4180d48a0 con 0x7fd41806d370 2024-03-08T08:30:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.987+0000 7fd4177fe640 1 --2- 172.21.15.105:0/1321270565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180d2660 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0097a0 tx=0x7fd40c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41765ee87ef485c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd416ffd640 1 -- 172.21.15.105:0/1321270565 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0315a0 con 0x7fd418161850 2024-03-08T08:30:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd416ffd640 1 -- 172.21.15.105:0/1321270565 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c037070 con 0x7fd418161850 2024-03-08T08:30:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd416ffd640 1 -- 172.21.15.105:0/1321270565 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c031960 con 0x7fd418161850 2024-03-08T08:30:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd41e484640 1 -- 172.21.15.105:0/1321270565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 msgr2=0x7fd4180d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd41e484640 1 --2- 172.21.15.105:0/1321270565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180d2660 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0097a0 tx=0x7fd40c02b840 comp rx=0 tx=0).stop 2024-03-08T08:30:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd41e484640 1 -- 172.21.15.105:0/1321270565 shutdown_connections 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd41e484640 1 --2- 172.21.15.105:0/1321270565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180d2660 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd41e484640 1 --2- 172.21.15.105:0/1321270565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 0x7fd41806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd41e484640 1 -- 172.21.15.105:0/1321270565 >> 172.21.15.105:0/1321270565 conn(0x7fd4180d3ce0 msgr2=0x7fd4180d40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd41e484640 1 -- 172.21.15.105:0/1321270565 shutdown_connections 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.988+0000 7fd41e484640 1 -- 172.21.15.105:0/1321270565 wait complete. 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.989+0000 7fd41e484640 1 Processor -- start 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41e484640 1 -- start start 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41e484640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd41806d370 0x7fd4180c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41e484640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 0x7fd418159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41e484640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41e484640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41806cf40 con 0x7fd418161850 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41e484640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41806bcb0 con 0x7fd41806d370 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41e484640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181f44c0 con 0x7fd418161470 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41ca21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41ca21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38456/0 (socket says 172.21.15.105:38456) 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.990+0000 7fd41ca21640 1 -- 172.21.15.105:0/3092390754 learned_addr learned my addr 172.21.15.105:0/3092390754 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd41ca21640 1 -- 172.21.15.105:0/3092390754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 msgr2=0x7fd418159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd41ca21640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 0x7fd418159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd41ca21640 1 -- 172.21.15.105:0/3092390754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd41806d370 msgr2=0x7fd4180c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd417fff640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd41806d370 0x7fd4180c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd41ca21640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd41806d370 0x7fd4180c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd41ca21640 1 -- 172.21.15.105:0/3092390754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4181f47a0 con 0x7fd418161850 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd41ca21640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180cef10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c7a0 tx=0x7fd41000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd414ff9640 1 -- 172.21.15.105:0/3092390754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410018070 con 0x7fd418161850 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd414ff9640 1 -- 172.21.15.105:0/3092390754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41000a640 con 0x7fd418161850 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd414ff9640 1 -- 172.21.15.105:0/3092390754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41000aab0 con 0x7fd418161850 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4180c78c0 con 0x7fd418161850 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.991+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4180b1350 con 0x7fd418161850 2024-03-08T08:30:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.992+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4181f4970 con 0x7fd418161850 2024-03-08T08:30:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.992+0000 7fd414ff9640 1 -- 172.21.15.105:0/3092390754 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd41001b070 con 0x7fd418161850 2024-03-08T08:30:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.993+0000 7fd414ff9640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd40406b4d0 0x7fd40406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.993+0000 7fd417fff640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd40406b4d0 0x7fd40406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.993+0000 7fd414ff9640 1 -- 172.21.15.105:0/3092390754 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd4100894b0 con 0x7fd418161850 2024-03-08T08:30:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.993+0000 7fd417fff640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd40406b4d0 0x7fd40406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd408006fd0 tx=0x7fd408008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:05.997+0000 7fd414ff9640 1 -- 172.21.15.105:0/3092390754 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4181f4970 con 0x7fd418161850 2024-03-08T08:30:06.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.169+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4180c76b0 con 0x7fd40406b4d0 2024-03-08T08:30:06.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.169+0000 7fd414ff9640 1 -- 172.21.15.105:0/3092390754 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd4180c76b0 con 0x7fd40406b4d0 2024-03-08T08:30:06.171 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 7 to deep-scrub 2024-03-08T08:30:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd40406b4d0 msgr2=0x7fd40406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd40406b4d0 0x7fd40406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd408006fd0 tx=0x7fd408008040 comp rx=0 tx=0).stop 2024-03-08T08:30:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 msgr2=0x7fd4180cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180cef10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c7a0 tx=0x7fd41000cc60 comp rx=0 tx=0).stop 2024-03-08T08:30:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 shutdown_connections 2024-03-08T08:30:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7fd40406b4d0 0x7fd40406d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd418161850 0x7fd4180cef10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd418161470 0x7fd418159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 --2- 172.21.15.105:0/3092390754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd41806d370 0x7fd4180c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 >> 172.21.15.105:0/3092390754 conn(0x7fd4180d3ce0 msgr2=0x7fd4180d40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 shutdown_connections 2024-03-08T08:30:06.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.171+0000 7fd41e484640 1 -- 172.21.15.105:0/3092390754 wait complete. 2024-03-08T08:30:06.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:30:06.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.282+0000 7f5afcbb5640 1 Processor -- start 2024-03-08T08:30:06.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.282+0000 7f5afcbb5640 1 -- start start 2024-03-08T08:30:06.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5afcbb5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5af81615d0 0x7f5af80b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5afcbb5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af80ceff0 0x7f5af80d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5afcbb5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af80cf660 con 0x7f5af80d05a0 2024-03-08T08:30:06.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5afcbb5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80d0f40 con 0x7f5af81615d0 2024-03-08T08:30:06.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5afcbb5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80cf940 con 0x7f5af80ceff0 2024-03-08T08:30:06.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5aeffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5af80d05a0 0x7f5af80ce8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:55836/0 (socket says 172.21.15.105:55836) 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5aeffff640 1 -- 172.21.15.105:0/3085744638 learned_addr learned my addr 172.21.15.105:0/3085744638 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538198736 0 0) 0x7f5af80cf660 con 0x7f5af80d05a0 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5af659c640 1 --2- 172.21.15.105:0/3085744638 >> v2:172.21.15.144:3300/0 conn(0x7f5af81615d0 0x7f5af80b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5af6d9d640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af80ceff0 0x7f5af80d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae4003520 con 0x7f5af80d05a0 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3357355109 0 0) 0x7f5ae4003520 con 0x7f5af80d05a0 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af80ceff0 msgr2=0x7f5af80d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.283+0000 7f5af5d9b640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af80ceff0 0x7f5af80d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 >> v2:172.21.15.144:3300/0 conn(0x7f5af81615d0 msgr2=0x7f5af80b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 --2- 172.21.15.105:0/3085744638 >> v2:172.21.15.144:3300/0 conn(0x7f5af81615d0 0x7f5af80b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80cfc20 con 0x7f5af80d05a0 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1533232777 0 0) 0x7f5ae0001670 con 0x7f5af80d05a0 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 >> v1:172.21.15.105:6789/0 conn(0x7f5af80d05a0 legacy=0x7f5af80ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae40039b0 0x7f5ae4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5ae4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af80ceff0 0x7f5ae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80cf940 con 0x7f5ae4004800 2024-03-08T08:30:06.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae400b9c0 con 0x7f5ae40039b0 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae400bb40 con 0x7f5af80ceff0 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5aeffff640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5ae4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af659c640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae40039b0 0x7f5ae4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af6d9d640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af80ceff0 0x7f5ae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5aeffff640 1 -- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af80ceff0 msgr2=0x7f5ae40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5aeffff640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af80ceff0 0x7f5ae40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5aeffff640 1 -- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae40039b0 msgr2=0x7f5ae4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5aeffff640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae40039b0 0x7f5ae4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5aeffff640 1 -- 172.21.15.105:0/3085744638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4003520 con 0x7f5ae4004800 2024-03-08T08:30:06.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.284+0000 7f5af659c640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae40039b0 0x7f5ae4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5aeffff640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5ae4004c80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0001050 tx=0x7f5ae4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f646a5f5d75ca9dd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0007d70 con 0x7f5ae4004800 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae0030070 con 0x7f5ae4004800 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5af5d9b640 1 -- 172.21.15.105:0/3085744638 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae002cd80 con 0x7f5ae4004800 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5afcbb5640 1 -- 172.21.15.105:0/3085744638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 msgr2=0x7f5ae4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5afcbb5640 1 --2- 172.21.15.105:0/3085744638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5ae4004c80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0001050 tx=0x7f5ae4005830 comp rx=0 tx=0).stop 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5af6d9d640 1 -- 172.21.15.105:0/3085744638 reap_dead start 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5afcbb5640 1 -- 172.21.15.105:0/3085744638 shutdown_connections 2024-03-08T08:30:06.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5afcbb5640 1 -- 172.21.15.105:0/3085744638 >> 172.21.15.105:0/3085744638 conn(0x7f5af809cfa0 msgr2=0x7f5af80d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:06.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5afcbb5640 1 -- 172.21.15.105:0/3085744638 shutdown_connections 2024-03-08T08:30:06.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.285+0000 7f5afcbb5640 1 -- 172.21.15.105:0/3085744638 wait complete. 2024-03-08T08:30:06.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.287+0000 7f5afcbb5640 1 Processor -- start 2024-03-08T08:30:06.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.287+0000 7f5afcbb5640 1 -- start start 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.287+0000 7f5afcbb5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5af8157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.287+0000 7f5afcbb5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af8157ac0 0x7f5af8159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5afcbb5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af81598d0 0x7f5af81f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5afcbb5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80b7060 con 0x7f5ae4004800 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5afcbb5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80d1c20 con 0x7f5af8157ac0 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5afcbb5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af81619b0 con 0x7f5af81598d0 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5af8157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5af8157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38468/0 (socket says 172.21.15.105:38468) 2024-03-08T08:30:06.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 -- 172.21.15.105:0/302301713 learned_addr learned my addr 172.21.15.105:0/302301713 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af6d9d640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af81598d0 0x7f5af81f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5aeffff640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af8157ac0 0x7f5af8159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 -- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af81598d0 msgr2=0x7f5af81f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af81598d0 0x7f5af81f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 -- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af8157ac0 msgr2=0x7f5af8159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af8157ac0 0x7f5af8159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 -- 172.21.15.105:0/302301713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af8159d80 con 0x7f5ae4004800 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af6d9d640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af81598d0 0x7f5af81f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:30:06.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5af659c640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5af8157580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80115d0 tx=0x7f5ae8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5aeeffd640 1 -- 172.21.15.105:0/302301713 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800f8f0 con 0x7f5ae4004800 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5aeeffd640 1 -- 172.21.15.105:0/302301713 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae800fa90 con 0x7f5ae4004800 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af815a050 con 0x7f5ae4004800 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5aeeffd640 1 -- 172.21.15.105:0/302301713 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800fe60 con 0x7f5ae4004800 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.288+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af80cc000 con 0x7f5ae4004800 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.289+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af80b4d50 con 0x7f5ae4004800 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.290+0000 7f5aeeffd640 1 -- 172.21.15.105:0/302301713 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ae8015460 con 0x7f5ae4004800 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.290+0000 7f5aeeffd640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5ad806b4d0 0x7f5ad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:06.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.290+0000 7f5aeeffd640 1 -- 172.21.15.105:0/302301713 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5ae808a8b0 con 0x7f5ae4004800 2024-03-08T08:30:06.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.290+0000 7f5aeffff640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5ad806b4d0 0x7f5ad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:06.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.291+0000 7f5aeffff640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5ad806b4d0 0x7f5ad806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0034000 tx=0x7f5ae0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:06.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.294+0000 7f5aeeffd640 1 -- 172.21.15.105:0/302301713 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5af80b4d50 con 0x7f5ae4004800 2024-03-08T08:30:06.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.461+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5af8075120 con 0x7f5ad806b4d0 2024-03-08T08:30:06.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.462+0000 7f5aeeffd640 1 -- 172.21.15.105:0/302301713 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13496 (secure 0 0 0) 0x7f5af8075120 con 0x7f5ad806b4d0 2024-03-08T08:30:06.463 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:30:06.463 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:30:06.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5ad806b4d0 msgr2=0x7f5ad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:06.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5ad806b4d0 0x7f5ad806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0034000 tx=0x7f5ae0007000 comp rx=0 tx=0).stop 2024-03-08T08:30:06.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 msgr2=0x7f5af8157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5af8157580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80115d0 tx=0x7f5ae8011a90 comp rx=0 tx=0).stop 2024-03-08T08:30:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 shutdown_connections 2024-03-08T08:30:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f5ad806b4d0 0x7f5ad806d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5af81598d0 0x7f5af81f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af8157ac0 0x7f5af8159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 --2- 172.21.15.105:0/302301713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ae4004800 0x7f5af8157580 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 >> 172.21.15.105:0/302301713 conn(0x7f5af809cfa0 msgr2=0x7f5af80b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:06.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 shutdown_connections 2024-03-08T08:30:06.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:06.464+0000 7f5afcbb5640 1 -- 172.21.15.105:0/302301713 wait complete. 2024-03-08T08:30:06.476 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-03-08T08:30:05.128141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":249720,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754725000,"statfs":{"total":773094113280,"available":772838400000,"internally_reserved":0,"allocated":4530176,"data_stored":2494792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":251055666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6806002,"num_objects":-1023,"num_object_clones":0,"num_object_copies":-2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-2130,"num_write_kb":-8652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001374"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-08T08:29:58.520095+0000","last_change":"2024-03-08T08:27:28.367794+0000","last_active":"2024-03-08T08:29:58.520095+0000","last_peered":"2024-03-08T08:29:58.520095+0000","last_clean":"2024-03-08T08:29:58.520095+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:29:58.520095+0000","last_undegraded":"2024-03-08T08:29:58.520095+0000","last_fullsized":"2024-03-08T08:29:58.520095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T08:27:27.135974+0000","last_clean_scrub_stamp":"2024-03-08T08:27:27.135974+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T19:35:29.822626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31592,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35752,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336088,"statfs":{"total":96636764160,"available":96600154112,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35748,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336092,"statfs":{"total":96636764160,"available":96600158208,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:30:06.478 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-08T08:27:27.135974+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=27, tm_wday=4, tm_yday=68, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=8, tm_hour=8, tm_min=30, tm_sec=1, tm_wday=4, tm_yday=68, tm_isdst=0) 2024-03-08T08:30:06.478 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-08T08:30:09.113 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:30:09.117 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:30:26.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T08:30:26.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.589+0000 7effcbe76640 1 Processor -- start 2024-03-08T08:30:26.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.590+0000 7effcbe76640 1 -- start start 2024-03-08T08:30:26.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effcbe76640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7effc4072440 0x7effc4071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effcbe76640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc415b670 con 0x7effc406e0a0 2024-03-08T08:30:26.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effcbe76640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc415b940 con 0x7effc40b4d00 2024-03-08T08:30:26.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effcbe76640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc415bc20 con 0x7effc4072440 2024-03-08T08:30:26.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effc9c12640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7effc4072440 0x7effc4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effca413640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7effc406e0a0 0x7effc40cdd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46434/0 (socket says 172.21.15.105:46434) 2024-03-08T08:30:26.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effca413640 1 -- 172.21.15.105:0/1354369125 learned_addr learned my addr 172.21.15.105:0/1354369125 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:26.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effc9c12640 1 -- 172.21.15.105:0/1354369125 >> v1:172.21.15.144:6789/0 conn(0x7effc40b4d00 legacy=0x7effc415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:26.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effc9c12640 1 -- 172.21.15.105:0/1354369125 >> v1:172.21.15.105:6789/0 conn(0x7effc406e0a0 legacy=0x7effc40cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effc8c10640 1 -- 172.21.15.105:0/1354369125 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851337004 0 0) 0x7effc415b670 con 0x7effc406e0a0 2024-03-08T08:30:26.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.591+0000 7effc9c12640 1 -- 172.21.15.105:0/1354369125 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4070840 con 0x7effc4072440 2024-03-08T08:30:26.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.592+0000 7effc9c12640 1 --2- 172.21.15.105:0/1354369125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc4072440 0x7effc4071020 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7effb40097a0 tx=0x7effb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e171b75f5bb857d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.592+0000 7effc8c10640 1 -- 172.21.15.105:0/1354369125 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4035020 con 0x7effc4072440 2024-03-08T08:30:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.592+0000 7effc8c10640 1 -- 172.21.15.105:0/1354369125 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb4031830 con 0x7effc4072440 2024-03-08T08:30:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.592+0000 7effc8c10640 1 -- 172.21.15.105:0/1354369125 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4031c00 con 0x7effc4072440 2024-03-08T08:30:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.593+0000 7effcbe76640 1 -- 172.21.15.105:0/1354369125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc4072440 msgr2=0x7effc4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.593+0000 7effcbe76640 1 --2- 172.21.15.105:0/1354369125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc4072440 0x7effc4071020 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7effb40097a0 tx=0x7effb402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.593+0000 7effcbe76640 1 -- 172.21.15.105:0/1354369125 shutdown_connections 2024-03-08T08:30:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.593+0000 7effcbe76640 1 --2- 172.21.15.105:0/1354369125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc4072440 0x7effc4071020 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.593+0000 7effcbe76640 1 -- 172.21.15.105:0/1354369125 >> 172.21.15.105:0/1354369125 conn(0x7effc40b1740 msgr2=0x7effc40b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.593+0000 7effcbe76640 1 -- 172.21.15.105:0/1354369125 shutdown_connections 2024-03-08T08:30:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.593+0000 7effcbe76640 1 -- 172.21.15.105:0/1354369125 wait complete. 2024-03-08T08:30:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.594+0000 7effcbe76640 1 Processor -- start 2024-03-08T08:30:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.594+0000 7effcbe76640 1 -- start start 2024-03-08T08:30:26.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.594+0000 7effcbe76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc406e0a0 0x7effc414d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effcbe76640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc4072440 0x7effc41e4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effcbe76640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effc40b4d00 0x7effc414b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effcbe76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4072820 con 0x7effc406e0a0 2024-03-08T08:30:26.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effcbe76640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4071560 con 0x7effc4072440 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effcbe76640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc40729a0 con 0x7effc40b4d00 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc406e0a0 0x7effc414d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc406e0a0 0x7effc414d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41284/0 (socket says 172.21.15.105:41284) 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 -- 172.21.15.105:0/3195790085 learned_addr learned my addr 172.21.15.105:0/3195790085 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 -- 172.21.15.105:0/3195790085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effc40b4d00 msgr2=0x7effc414b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effc40b4d00 0x7effc414b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 -- 172.21.15.105:0/3195790085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc4072440 msgr2=0x7effc41e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc4072440 0x7effc41e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 -- 172.21.15.105:0/3195790085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc414bbc0 con 0x7effc406e0a0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effc9c12640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc406e0a0 0x7effc414d250 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7effb40098c0 tx=0x7effb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effba7fc640 1 -- 172.21.15.105:0/3195790085 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb403ba80 con 0x7effc406e0a0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effba7fc640 1 -- 172.21.15.105:0/3195790085 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb403bc20 con 0x7effc406e0a0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effba7fc640 1 -- 172.21.15.105:0/3195790085 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb40453f0 con 0x7effc406e0a0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc414bea0 con 0x7effc406e0a0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.595+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc4146ae0 con 0x7effc406e0a0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.596+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc4075120 con 0x7effc406e0a0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.597+0000 7effba7fc640 1 -- 172.21.15.105:0/3195790085 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effb404c060 con 0x7effc406e0a0 2024-03-08T08:30:26.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.597+0000 7effba7fc640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effa806b4d0 0x7effa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.597+0000 7effba7fc640 1 -- 172.21.15.105:0/3195790085 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7effb40abbd0 con 0x7effc406e0a0 2024-03-08T08:30:26.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.597+0000 7effc9411640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effa806b4d0 0x7effa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.597+0000 7effc9411640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effa806b4d0 0x7effa806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7effc40b5130 tx=0x7effc0006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:26.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.601+0000 7effba7fc640 1 -- 172.21.15.105:0/3195790085 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7effb4040070 con 0x7effc406e0a0 2024-03-08T08:30:26.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.768+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 --> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effc4151a70 con 0x7effa806b4d0 2024-03-08T08:30:26.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.769+0000 7effba7fc640 1 -- 172.21.15.105:0/3195790085 <== mgr.4107 v2:172.21.15.105:6832/3146407311 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13357 (secure 0 0 0) 0x7effc4151a70 con 0x7effa806b4d0 2024-03-08T08:30:26.770 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:30:26.770 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-08T08:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effa806b4d0 msgr2=0x7effa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effa806b4d0 0x7effa806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7effc40b5130 tx=0x7effc0006ba0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc406e0a0 msgr2=0x7effc414d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc406e0a0 0x7effc414d250 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7effb40098c0 tx=0x7effb40046c0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 shutdown_connections 2024-03-08T08:30:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7effa806b4d0 0x7effa806d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effc40b4d00 0x7effc414b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc4072440 0x7effc41e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 --2- 172.21.15.105:0/3195790085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc406e0a0 0x7effc414d250 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 >> 172.21.15.105:0/3195790085 conn(0x7effc40b1740 msgr2=0x7effc4150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 shutdown_connections 2024-03-08T08:30:26.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.771+0000 7effcbe76640 1 -- 172.21.15.105:0/3195790085 wait complete. 2024-03-08T08:30:26.782 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-03-08T08:30:25.132759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":249704,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754725016,"statfs":{"total":773094113280,"available":772838416384,"internally_reserved":0,"allocated":4530176,"data_stored":2494792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":251055666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003113"},"pg_stats":[{"pgid":"1.0","version":"36'119","reported_seq":212,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-08T08:30:07.102221+0000","last_change":"2024-03-08T08:30:07.102258+0000","last_active":"2024-03-08T08:30:07.102221+0000","last_peered":"2024-03-08T08:30:07.102221+0000","last_clean":"2024-03-08T08:30:07.102221+0000","last_became_active":"2024-03-08T08:27:28.366669+0000","last_became_peered":"2024-03-08T08:27:28.366669+0000","last_unstale":"2024-03-08T08:30:07.102221+0000","last_undegraded":"2024-03-08T08:30:07.102221+0000","last_fullsized":"2024-03-08T08:30:07.102221+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'119","last_scrub_stamp":"2024-03-08T08:30:07.102140+0000","last_deep_scrub":"36'119","last_deep_scrub_stamp":"2024-03-08T08:30:07.102140+0000","last_clean_scrub_stamp":"2024-03-08T08:30:07.102140+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T17:32:32.373364+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31592,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35752,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336088,"statfs":{"total":96636764160,"available":96600154112,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35748,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336092,"statfs":{"total":96636764160,"available":96600158208,"internally_reserved":0,"allocated":417792,"data_stored":164257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1011712,"data_stored":754625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T08:30:26.783 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-08T08:30:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.882+0000 7f4b7709d640 1 Processor -- start 2024-03-08T08:30:26.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.882+0000 7f4b7709d640 1 -- start start 2024-03-08T08:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b7709d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b70072440 0x7f4b70071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b7709d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b700b4d00 0x7f4b7015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b7709d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4b7015b4a0 0x7f4b7006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b7709d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7015b940 con 0x7f4b700b4d00 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b7709d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7015bc20 con 0x7f4b7015b4a0 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b7709d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b70070840 con 0x7f4b70072440 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b6ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b700b4d00 0x7f4b7015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b6ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b700b4d00 0x7f4b7015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41298/0 (socket says 172.21.15.105:41298) 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b6ffff640 1 -- 172.21.15.105:0/2436550731 learned_addr learned my addr 172.21.15.105:0/2436550731 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b74e39640 1 --2- 172.21.15.105:0/2436550731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b70072440 0x7f4b70071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b7563a640 1 --2- 172.21.15.105:0/2436550731 >> v2:172.21.15.144:3300/0 conn(0x7f4b7015b4a0 0x7f4b7006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b6ffff640 1 -- 172.21.15.105:0/2436550731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b70072440 msgr2=0x7f4b70071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b6ffff640 1 --2- 172.21.15.105:0/2436550731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b70072440 0x7f4b70071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b6ffff640 1 -- 172.21.15.105:0/2436550731 >> v2:172.21.15.144:3300/0 conn(0x7f4b7015b4a0 msgr2=0x7f4b7006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.883+0000 7f4b6ffff640 1 --2- 172.21.15.105:0/2436550731 >> v2:172.21.15.144:3300/0 conn(0x7f4b7015b4a0 0x7f4b7006e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.884+0000 7f4b6ffff640 1 -- 172.21.15.105:0/2436550731 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b70070b20 con 0x7f4b700b4d00 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.884+0000 7f4b74e39640 1 --2- 172.21.15.105:0/2436550731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b70072440 0x7f4b70071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.884+0000 7f4b6ffff640 1 --2- 172.21.15.105:0/2436550731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b700b4d00 0x7f4b7015af60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4b70158aa0 tx=0x7f4b6402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b71f4491f27192 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.884+0000 7f4b6f7fe640 1 -- 172.21.15.105:0/2436550731 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6403b070 con 0x7f4b700b4d00 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.884+0000 7f4b6f7fe640 1 -- 172.21.15.105:0/2436550731 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b64036440 con 0x7f4b700b4d00 2024-03-08T08:30:26.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.884+0000 7f4b6f7fe640 1 -- 172.21.15.105:0/2436550731 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64036810 con 0x7f4b700b4d00 2024-03-08T08:30:26.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 -- 172.21.15.105:0/2436550731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b700b4d00 msgr2=0x7f4b7015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 --2- 172.21.15.105:0/2436550731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b700b4d00 0x7f4b7015af60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4b70158aa0 tx=0x7f4b6402e0e0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 -- 172.21.15.105:0/2436550731 shutdown_connections 2024-03-08T08:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 --2- 172.21.15.105:0/2436550731 >> v2:172.21.15.144:3300/0 conn(0x7f4b7015b4a0 0x7f4b7006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 --2- 172.21.15.105:0/2436550731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b700b4d00 0x7f4b7015af60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 --2- 172.21.15.105:0/2436550731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b70072440 0x7f4b70071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 -- 172.21.15.105:0/2436550731 >> 172.21.15.105:0/2436550731 conn(0x7f4b700b1740 msgr2=0x7f4b700b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 -- 172.21.15.105:0/2436550731 shutdown_connections 2024-03-08T08:30:26.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.885+0000 7f4b7709d640 1 -- 172.21.15.105:0/2436550731 wait complete. 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.886+0000 7f4b7709d640 1 Processor -- start 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.886+0000 7f4b7709d640 1 -- start start 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b7709d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b70072440 0x7f4b70145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b7709d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b700b4d00 0x7f4b7014e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b7709d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b7015b4a0 0x7f4b701575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b7709d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b700729a0 con 0x7f4b70072440 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b7709d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b70072820 con 0x7f4b7015b4a0 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b7709d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7006e5e0 con 0x7f4b700b4d00 2024-03-08T08:30:26.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b70072440 0x7f4b70145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b70072440 0x7f4b70145e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41302/0 (socket says 172.21.15.105:41302) 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 -- 172.21.15.105:0/1354177622 learned_addr learned my addr 172.21.15.105:0/1354177622 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b6ffff640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b700b4d00 0x7f4b7014e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b7563a640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b7015b4a0 0x7f4b701575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 -- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b700b4d00 msgr2=0x7f4b7014e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b700b4d00 0x7f4b7014e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 -- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b7015b4a0 msgr2=0x7f4b701575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b7015b4a0 0x7f4b701575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:26.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 -- 172.21.15.105:0/1354177622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7014f9d0 con 0x7f4b70072440 2024-03-08T08:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b7563a640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b7015b4a0 0x7f4b701575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T08:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.887+0000 7f4b74e39640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b70072440 0x7f4b70145e90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000e730 tx=0x7f4b6000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.888+0000 7f4b6d7fa640 1 -- 172.21.15.105:0/1354177622 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000f040 con 0x7f4b70072440 2024-03-08T08:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.888+0000 7f4b6d7fa640 1 -- 172.21.15.105:0/1354177622 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60004230 con 0x7f4b70072440 2024-03-08T08:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.888+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b70156ad0 con 0x7f4b70072440 2024-03-08T08:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.888+0000 7f4b6d7fa640 1 -- 172.21.15.105:0/1354177622 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000ee40 con 0x7f4b70072440 2024-03-08T08:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.888+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7014e8a0 con 0x7f4b70072440 2024-03-08T08:30:26.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.889+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b70075120 con 0x7f4b70072440 2024-03-08T08:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.889+0000 7f4b6d7fa640 1 -- 172.21.15.105:0/1354177622 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b6000c6d0 con 0x7f4b70072440 2024-03-08T08:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.889+0000 7f4b6d7fa640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4b5c06b500 0x7f4b5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.889+0000 7f4b6d7fa640 1 -- 172.21.15.105:0/1354177622 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4b60089bb0 con 0x7f4b70072440 2024-03-08T08:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.889+0000 7f4b6ffff640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4b5c06b500 0x7f4b5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.890+0000 7f4b6ffff640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4b5c06b500 0x7f4b5c06d9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b64038000 tx=0x7f4b64032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:26.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:26.894+0000 7f4b6d7fa640 1 -- 172.21.15.105:0/1354177622 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b60017020 con 0x7f4b70072440 2024-03-08T08:30:27.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.064+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f4b700b1f30 con 0x7f4b70072440 2024-03-08T08:30:27.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.068+0000 7f4b6d7fa640 1 -- 172.21.15.105:0/1354177622 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4b60058dc0 con 0x7f4b70072440 2024-03-08T08:30:27.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.068+0000 7f4b6d7fa640 1 -- 172.21.15.105:0/1354177622 <== mon.0 v2:172.21.15.105:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4b600586c0 con 0x7f4b70072440 2024-03-08T08:30:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4b5c06b500 msgr2=0x7f4b5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4b5c06b500 0x7f4b5c06d9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b64038000 tx=0x7f4b64032000 comp rx=0 tx=0).stop 2024-03-08T08:30:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b70072440 msgr2=0x7f4b70145e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b70072440 0x7f4b70145e90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000e730 tx=0x7f4b6000ebf0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 shutdown_connections 2024-03-08T08:30:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f4b5c06b500 0x7f4b5c06d9c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b7015b4a0 0x7f4b701575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b700b4d00 0x7f4b7014e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 --2- 172.21.15.105:0/1354177622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b70072440 0x7f4b70145e90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 >> 172.21.15.105:0/1354177622 conn(0x7f4b700b1740 msgr2=0x7f4b7015e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 shutdown_connections 2024-03-08T08:30:27.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.070+0000 7f4b7709d640 1 -- 172.21.15.105:0/1354177622 wait complete. 2024-03-08T08:30:27.082 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-08T08:30:27.083 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T08:30:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.181+0000 7f2d9590c640 1 Processor -- start 2024-03-08T08:30:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.181+0000 7f2d9590c640 1 -- start start 2024-03-08T08:30:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d9590c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d900cf290 0x7f2d900b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d9590c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d900b4920 con 0x7f2d900b4000 2024-03-08T08:30:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d9590c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d90072120 con 0x7f2d9006cf90 2024-03-08T08:30:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d9590c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d900cd9e0 con 0x7f2d900cf290 2024-03-08T08:30:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d8e7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d900cf290 0x7f2d900b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d8effd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2d900b4000 0x7f2d9015b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46448/0 (socket says 172.21.15.105:46448) 2024-03-08T08:30:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d8effd640 1 -- 172.21.15.105:0/508302820 learned_addr learned my addr 172.21.15.105:0/508302820 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-03-08T08:30:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d8e7fc640 1 -- 172.21.15.105:0/508302820 >> v1:172.21.15.144:6789/0 conn(0x7f2d9006cf90 legacy=0x7f2d900cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T08:30:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d8e7fc640 1 -- 172.21.15.105:0/508302820 >> v1:172.21.15.105:6789/0 conn(0x7f2d900b4000 legacy=0x7f2d9015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d8e7fc640 1 -- 172.21.15.105:0/508302820 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90072400 con 0x7f2d900cf290 2024-03-08T08:30:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.182+0000 7f2d8dffb640 1 -- 172.21.15.105:0/508302820 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576098128 0 0) 0x7f2d900b4920 con 0x7f2d900b4000 2024-03-08T08:30:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.183+0000 7f2d8e7fc640 1 --2- 172.21.15.105:0/508302820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900cf290 0x7f2d900b43e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2d840097a0 tx=0x7f2d8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d48f7d9cdee9fe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.183+0000 7f2d8dffb640 1 -- 172.21.15.105:0/508302820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84035020 con 0x7f2d900cf290 2024-03-08T08:30:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.183+0000 7f2d8dffb640 1 -- 172.21.15.105:0/508302820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2d84031830 con 0x7f2d900cf290 2024-03-08T08:30:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.183+0000 7f2d8dffb640 1 -- 172.21.15.105:0/508302820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84031c20 con 0x7f2d900cf290 2024-03-08T08:30:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.184+0000 7f2d9590c640 1 -- 172.21.15.105:0/508302820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900cf290 msgr2=0x7f2d900b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.184+0000 7f2d9590c640 1 --2- 172.21.15.105:0/508302820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900cf290 0x7f2d900b43e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2d840097a0 tx=0x7f2d8402b8d0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.184+0000 7f2d9590c640 1 -- 172.21.15.105:0/508302820 shutdown_connections 2024-03-08T08:30:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.184+0000 7f2d9590c640 1 --2- 172.21.15.105:0/508302820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900cf290 0x7f2d900b43e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.184+0000 7f2d9590c640 1 -- 172.21.15.105:0/508302820 >> 172.21.15.105:0/508302820 conn(0x7f2d900b1740 msgr2=0x7f2d900b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.184+0000 7f2d9590c640 1 -- 172.21.15.105:0/508302820 shutdown_connections 2024-03-08T08:30:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.184+0000 7f2d9590c640 1 -- 172.21.15.105:0/508302820 wait complete. 2024-03-08T08:30:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.185+0000 7f2d9590c640 1 Processor -- start 2024-03-08T08:30:27.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.185+0000 7f2d9590c640 1 -- start start 2024-03-08T08:30:27.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d9590c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9006cf90 0x7f2d9015d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:27.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d9590c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900b4000 0x7f2d90141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:27.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d9590c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d900cf290 0x7f2d901423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d9590c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9015a500 con 0x7f2d900b4000 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d9590c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d900726e0 con 0x7f2d9006cf90 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d9590c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9006d400 con 0x7f2d900cf290 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900b4000 0x7f2d90141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900b4000 0x7f2d90141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41328/0 (socket says 172.21.15.105:41328) 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 -- 172.21.15.105:0/3221804157 learned_addr learned my addr 172.21.15.105:0/3221804157 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 -- 172.21.15.105:0/3221804157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d900cf290 msgr2=0x7f2d901423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d900cf290 0x7f2d901423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 -- 172.21.15.105:0/3221804157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9006cf90 msgr2=0x7f2d9015d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9006cf90 0x7f2d9015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 -- 172.21.15.105:0/3221804157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d90140b50 con 0x7f2d900b4000 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d8e7fc640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900b4000 0x7f2d90141e90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2d84000c00 tx=0x7f2d840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d77fff640 1 -- 172.21.15.105:0/3221804157 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84041070 con 0x7f2d900b4000 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d77fff640 1 -- 172.21.15.105:0/3221804157 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2d8403a9f0 con 0x7f2d900b4000 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d90140e30 con 0x7f2d900b4000 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d77fff640 1 -- 172.21.15.105:0/3221804157 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84035040 con 0x7f2d900b4000 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.186+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9015adc0 con 0x7f2d900b4000 2024-03-08T08:30:27.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.187+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d90075120 con 0x7f2d900b4000 2024-03-08T08:30:27.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.188+0000 7f2d77fff640 1 -- 172.21.15.105:0/3221804157 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d8404c060 con 0x7f2d900b4000 2024-03-08T08:30:27.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.188+0000 7f2d77fff640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T08:30:27.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.188+0000 7f2d77fff640 1 -- 172.21.15.105:0/3221804157 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2d840aba00 con 0x7f2d900b4000 2024-03-08T08:30:27.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.188+0000 7f2d8effd640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T08:30:27.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.188+0000 7f2d8effd640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d80001560 tx=0x7f2d80007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T08:30:27.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.192+0000 7f2d77fff640 1 -- 172.21.15.105:0/3221804157 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d8407ab80 con 0x7f2d900b4000 2024-03-08T08:30:27.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.400+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d90067fc0 con 0x7f2d900b4000 2024-03-08T08:30:27.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.401+0000 7f2d77fff640 1 -- 172.21.15.105:0/3221804157 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f2d8407a480 con 0x7f2d900b4000 2024-03-08T08:30:27.402 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-08T08:30:27.402 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2759286099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2759286099},{"type":"v1","addr":"172.21.15.144:6834","nonce":2759286099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2788852991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2788852991},{"type":"v1","addr":"172.21.15.144:6835","nonce":2788852991}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2226664928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2226664928},{"type":"v1","addr":"172.21.15.105:6835","nonce":2226664928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3047736114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3047736114},{"type":"v1","addr":"172.21.15.105:6837","nonce":3047736114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[]} 2024-03-08T08:30:27.403 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 15 2024-03-08T08:30:27.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2d7c06b4d0 msgr2=0x7f2d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:27.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d80001560 tx=0x7f2d80007ca0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900b4000 msgr2=0x7f2d90141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T08:30:27.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900b4000 0x7f2d90141e90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2d84000c00 tx=0x7f2d840046c0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 shutdown_connections 2024-03-08T08:30:27.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:6832/3146407311,v1:172.21.15.105:6833/3146407311] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d900cf290 0x7f2d901423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d900b4000 0x7f2d90141e90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 --2- 172.21.15.105:0/3221804157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9006cf90 0x7f2d9015d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T08:30:27.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 >> 172.21.15.105:0/3221804157 conn(0x7f2d900b1740 msgr2=0x7f2d90157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T08:30:27.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 shutdown_connections 2024-03-08T08:30:27.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-08T08:30:27.403+0000 7f2d9590c640 1 -- 172.21.15.105:0/3221804157 wait complete. 2024-03-08T08:30:27.415 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-08T08:30:27.415 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-08T08:30:27.415 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-08T08:30:27.415 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.423 INFO:tasks.ceph.mds.a:Stopped 2024-03-08T08:30:27.424 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-08T08:30:27.424 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.473 INFO:tasks.ceph.mds.c:Stopped 2024-03-08T08:30:27.473 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-08T08:30:27.474 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.486 INFO:tasks.ceph.mds.b:Stopped 2024-03-08T08:30:27.486 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-08T08:30:27.486 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.538 INFO:tasks.ceph.mds.d:Stopped 2024-03-08T08:30:27.538 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-08T08:30:27.539 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-08T08:30:27.539 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.570 INFO:tasks.ceph.osd.0:Stopped 2024-03-08T08:30:27.571 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-08T08:30:27.571 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.644 INFO:tasks.ceph.osd.1:Stopped 2024-03-08T08:30:27.644 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-08T08:30:27.645 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.711 INFO:tasks.ceph.osd.2:Stopped 2024-03-08T08:30:27.711 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-08T08:30:27.711 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.777 INFO:tasks.ceph.osd.3:Stopped 2024-03-08T08:30:27.778 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-08T08:30:27.778 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.818 INFO:tasks.ceph.osd.4:Stopped 2024-03-08T08:30:27.818 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-08T08:30:27.818 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.859 INFO:tasks.ceph.osd.5:Stopped 2024-03-08T08:30:27.859 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-08T08:30:27.859 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.927 INFO:tasks.ceph.osd.6:Stopped 2024-03-08T08:30:27.928 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-08T08:30:27.928 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:27.999 INFO:tasks.ceph.osd.7:Stopped 2024-03-08T08:30:27.999 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-08T08:30:28.000 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-08T08:30:28.000 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:28.025 INFO:tasks.ceph.mgr.y:Stopped 2024-03-08T08:30:28.026 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-08T08:30:28.026 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:28.067 INFO:tasks.ceph.mgr.x:Stopped 2024-03-08T08:30:28.067 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-08T08:30:28.067 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-08T08:30:28.068 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:28.078 INFO:tasks.ceph.mon.a:Stopped 2024-03-08T08:30:28.078 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-08T08:30:28.078 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:28.120 INFO:tasks.ceph.mon.b:Stopped 2024-03-08T08:30:28.121 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-08T08:30:28.121 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T08:30:28.173 INFO:tasks.ceph.mon.c:Stopped 2024-03-08T08:30:28.173 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-08T08:30:28.173 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'missing required features' | head -n 1 2024-03-08T08:30:28.208 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:30:28.208 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-08T08:30:28.402 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:30:28.403 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-08T08:30:28.506 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:30:28.507 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-08T08:30:28.573 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:30:28.573 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-08T08:30:28.632 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:30:28.632 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-08T08:30:28.794 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:30:28.794 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-08T08:30:28.858 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:30:28.859 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-08T08:30:28.917 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:30:28.917 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-08T08:30:28.973 INFO:tasks.ceph:Archiving mon data... 2024-03-08T08:30:28.974 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/data/mon.a.tgz 2024-03-08T08:30:28.974 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-03-08T08:30:28.989 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.MUrcTPtswf 2024-03-08T08:30:28.990 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.MUrcTPtswf 2024-03-08T08:30:29.119 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.MUrcTPtswf 2024-03-08T08:30:29.205 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.MUrcTPtswf is 284KB 2024-03-08T08:30:29.303 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.MUrcTPtswf 2024-03-08T08:30:29.318 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/data/mon.b.tgz 2024-03-08T08:30:29.318 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-03-08T08:30:29.334 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.faRtFIogFd 2024-03-08T08:30:29.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.faRtFIogFd 2024-03-08T08:30:29.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.faRtFIogFd 2024-03-08T08:30:29.568 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.faRtFIogFd is 381KB 2024-03-08T08:30:29.664 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.faRtFIogFd 2024-03-08T08:30:29.679 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/data/mon.c.tgz 2024-03-08T08:30:29.679 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-03-08T08:30:29.732 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.QGJ6Q9LNp8 2024-03-08T08:30:29.732 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.QGJ6Q9LNp8 2024-03-08T08:30:29.868 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.QGJ6Q9LNp8 2024-03-08T08:30:29.952 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.QGJ6Q9LNp8 is 381KB 2024-03-08T08:30:30.048 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.QGJ6Q9LNp8 2024-03-08T08:30:30.062 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-08T08:30:30.063 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-08T08:30:30.065 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-08T08:30:30.147 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-08T08:30:30.182 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-08T08:30:30.183 INFO:tasks.ceph:Archiving crash dumps... 2024-03-08T08:30:30.184 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/remote/smithi105/crash 2024-03-08T08:30:30.185 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-08T08:30:30.215 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/remote/smithi144/crash 2024-03-08T08:30:30.216 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-08T08:30:30.248 INFO:tasks.ceph:Shutting down logrotate 2024-03-08T08:30:30.249 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:30:30.260 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-08T08:30:30.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-08T08:30:30.361 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-08T08:30:30.388 INFO:tasks.ceph:Compressing logs... 2024-03-08T08:30:30.388 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T08:30:30.404 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T08:30:30.488 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-08T08:30:30.488 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37891.log 2024-03-08T08:30:30.488 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44920.log 2024-03-08T08:30:30.488 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.37891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37971.log 2024-03-08T08:30:30.488 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.44920.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37891.log.gz 2024-03-08T08:30:30.488 INFO:teuthology.orchestra.run.smithi144.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44920.log.gz 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37771.log 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.37971.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.37771.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37971.log.gz 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37771.log.gz 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37931.log 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37811.log 2024-03-08T08:30:30.489 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.37931.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-08T08:30:30.490 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.37811.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-08T08:30:30.490 INFO:teuthology.orchestra.run.smithi144.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37931.log.gz 2024-03-08T08:30:30.490 INFO:teuthology.orchestra.run.smithi144.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-08T08:30:30.490 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38011.log 2024-03-08T08:30:30.490 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-08T08:30:30.490 INFO:teuthology.orchestra.run.smithi144.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37811.log.gz 2024-03-08T08:30:30.490 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-08T08:30:30.491 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.38011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44311.log 2024-03-08T08:30:30.491 INFO:teuthology.orchestra.run.smithi144.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38011.log.gz 2024-03-08T08:30:30.491 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-08T08:30:30.491 INFO:teuthology.orchestra.run.smithi144.stderr:gzip/var/log/ceph/ceph-client.3.44311.log: -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.3.44311.log.gz 2024-03-08T08:30:30.491 INFO:teuthology.orchestra.run.smithi144.stderr: -- /var/log/ceph/ceph-client.admin.37731.log 2024-03-08T08:30:30.491 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-08T08:30:30.491 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.37731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37851.log 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2024-03-08T08:30:30.491 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-08T08:30:30.492 INFO:teuthology.orchestra.run.smithi144.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-08T08:30:30.492 INFO:teuthology.orchestra.run.smithi144.stderr:gzip/var/log/ceph/ceph-mon.b.log: -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-08T08:30:30.492 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.37851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43922.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37851.log.gz 2024-03-08T08:30:30.492 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-08T08:30:30.492 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-08T08:30:30.492 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.43922.log: /var/log/ceph/ceph-mds.d.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43922.log.gz 2024-03-08T08:30:30.492 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mon.c.log: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-08T08:30:30.515 INFO:teuthology.orchestra.run.smithi144.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-08T08:30:30.536 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54010.log 2024-03-08T08:30:30.536 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-08T08:30:30.536 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44917.log 2024-03-08T08:30:30.536 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54010.log: gzip/var/log/ceph/ceph-mon.a.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48513.log 2024-03-08T08:30:30.537 INFO:teuthology.orchestra.run.smithi105.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54010.log.gz 2024-03-08T08:30:30.537 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43860.log 2024-03-08T08:30:30.537 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48513.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44917.log.gz 2024-03-08T08:30:30.537 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53110.log 2024-03-08T08:30:30.537 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48513.log.gz 2024-03-08T08:30:30.537 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/ceph-client.admin.43860.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49059.log 2024-03-08T08:30:30.537 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53320.log 2024-03-08T08:30:30.537 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43860.log.gz 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49059.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53110.log.gz 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52312.log 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49059.log.gz 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51030.log 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44585.log 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53320.log.gz 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51030.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52312.log.gz 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50019.log 2024-03-08T08:30:30.538 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51030.log.gz 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45258.log 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47844.log 88.1% 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44585.log.gz 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45258.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50019.log.gz 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51864.log 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45258.log.gz 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50324.log 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51864.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47844.log.gzgzip 2024-03-08T08:30:30.539 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52648.log 2024-03-08T08:30:30.540 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45139.log 2024-03-08T08:30:30.540 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51864.log.gz 2024-03-08T08:30:30.540 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50324.log.gz 2024-03-08T08:30:30.540 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48897.log 2024-03-08T08:30:30.540 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52648.log.gz 2024-03-08T08:30:30.540 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-03-08T08:30:30.540 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48897.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45139.log.gz 2024-03-08T08:30:30.540 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54092.log 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48897.log.gz 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48729.log 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54092.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gzgzip 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53572.log 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48729.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54092.log.gz 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50802.log 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48729.log.gz 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48072.log 2024-03-08T08:30:30.541 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53572.log.gz 2024-03-08T08:30:30.542 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43944.log 2024-03-08T08:30:30.542 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49521.log 2024-03-08T08:30:30.542 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50802.log.gz 2024-03-08T08:30:30.542 INFO:teuthology.orchestra.run.smithi105.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48072.log.gz 2024-03-08T08:30:30.542 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43714.log 2024-03-08T08:30:30.542 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43944.log.gz 2024-03-08T08:30:30.542 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49563.log 2024-03-08T08:30:30.542 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43714.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49521.log.gz -5 2024-03-08T08:30:30.543 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45750.log 2024-03-08T08:30:30.543 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43714.log.gz 2024-03-08T08:30:30.543 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52270.log 2024-03-08T08:30:30.543 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49563.log.gz 2024-03-08T08:30:30.543 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52564.log 2024-03-08T08:30:30.543 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37362.log 2024-03-08T08:30:30.543 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45750.log.gz 2024-03-08T08:30:30.543 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52564.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52270.log.gz 2024-03-08T08:30:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50844.log 2024-03-08T08:30:30.544 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52564.log.gz 2024-03-08T08:30:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47340.log 2024-03-08T08:30:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50844.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37362.log.gz 2024-03-08T08:30:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2024-03-08T08:30:30.544 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50844.log.gz 2024-03-08T08:30:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48471.log 2024-03-08T08:30:30.544 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44070.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47340.log.gz 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45624.log 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48471.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45624.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48471.log.gz 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50718.log 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45624.log.gz 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53824.log 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50718.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-03-08T08:30:30.545 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46086.log 2024-03-08T08:30:30.546 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50718.log.gz 2024-03-08T08:30:30.546 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46700.log 2024-03-08T08:30:30.546 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53824.log.gz 2024-03-08T08:30:30.546 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50061.log 2024-03-08T08:30:30.546 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49185.log 2024-03-08T08:30:30.546 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46700.log.gz 2024-03-08T08:30:30.546 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50061.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46086.log.gz 2024-03-08T08:30:30.546 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47988.log 2024-03-08T08:30:30.547 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50061.log.gz 2024-03-08T08:30:30.547 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 88.2% 2024-03-08T08:30:30.547 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49185.log.gz 2024-03-08T08:30:30.547 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49017.log 2024-03-08T08:30:30.547 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47988.log: /var/log/ceph/ceph-client.admin.47928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50188.log 2024-03-08T08:30:30.547 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47988.log.gz 2024-03-08T08:30:30.547 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49017.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gz 2024-03-08T08:30:30.547 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50145.log 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50188.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49017.log.gz 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50103.log 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50145.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50188.log.gz 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47424.log 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50103.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50145.log.gz 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43902.log 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47424.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50103.log.gz 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53278.log 2024-03-08T08:30:30.548 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47424.log.gz 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48393.log 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53278.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43902.log.gz 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47802.log 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53278.log.gz 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48597.log 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48393.log.gz 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48030.log 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48597.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47802.log.gz 2024-03-08T08:30:30.549 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48030.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48597.log.gz 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53488.log 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48030.log.gz 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50464.log 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53488.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45666.log 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50464.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53488.log.gz 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44912.log 2024-03-08T08:30:30.550 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50464.log.gz 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44912.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45666.log.gz 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43544.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44912.log.gz 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48975.log 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44913.log 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2024-03-08T08:30:30.551 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44913.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48975.log.gz 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45250.log 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44913.log.gz 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44280.log 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45250.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52396.log 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44280.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45250.log.gz 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45256.log 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44280.log.gz 2024-03-08T08:30:30.552 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52522.log 2024-03-08T08:30:30.553 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45256.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52396.log.gz 2024-03-08T08:30:30.553 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43670.log 2024-03-08T08:30:30.553 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52522.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45256.log.gz 2024-03-08T08:30:30.553 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44582.log 2024-03-08T08:30:30.553 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52522.log.gz 2024-03-08T08:30:30.553 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47466.log 2024-03-08T08:30:30.553 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44582.log: gzip -5 --verbose 88.3% -- /var/log/ceph/ceph-client.admin.44579.log -- replaced with /var/log/ceph/ceph-client.admin.43670.log.gz 2024-03-08T08:30:30.553 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:30.554 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47466.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44582.log.gz 2024-03-08T08:30:30.554 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51643.log 2024-03-08T08:30:30.554 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47466.log.gz 2024-03-08T08:30:30.554 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44579.log: /var/log/ceph/ceph-client.admin.47676.log 2024-03-08T08:30:30.554 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49731.log 2024-03-08T08:30:30.554 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44579.log.gz 2024-03-08T08:30:30.554 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47676.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-08T08:30:30.554 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47676.log.gz 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52835.log 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51643.log.gz 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.49731.log.gz 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45918.log 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45918.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52835.log.gz 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44587.log 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45918.log.gz 2024-03-08T08:30:30.555 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50946.log 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53614.log 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44587.log: /var/log/ceph/ceph-client.admin.50946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53908.log 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44587.log.gz 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53614.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.50946.log.gz 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44322.log 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53908.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53614.log.gz 2024-03-08T08:30:30.556 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46568.log 2024-03-08T08:30:30.557 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53908.log.gz 2024-03-08T08:30:30.557 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37112.log 2024-03-08T08:30:30.557 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.46568.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.45456.log 2024-03-08T08:30:30.557 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44322.log.gz 2024-03-08T08:30:30.557 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46568.log.gz 2024-03-08T08:30:30.557 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53194.log 2024-03-08T08:30:30.557 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51072.log 89.6% 2024-03-08T08:30:30.557 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37112.log.gz 2024-03-08T08:30:30.558 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53194.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.45456.log.gz -5 2024-03-08T08:30:30.558 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37481.log 2024-03-08T08:30:30.558 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47130.log 2024-03-08T08:30:30.558 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53194.log.gz 2024-03-08T08:30:30.558 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47256.log 2024-03-08T08:30:30.558 INFO:teuthology.orchestra.run.smithi105.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51072.log.gz 2024-03-08T08:30:30.558 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47130.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37481.log.gz 2024-03-08T08:30:30.558 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53950.log 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47130.log.gz 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49311.log 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53950.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47256.log.gz 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50413.log 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49311.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53950.log.gz 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53236.log 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50413.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.53362.log -- replaced with /var/log/ceph/ceph-client.admin.49311.log.gz 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53236.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50413.log.gz 2024-03-08T08:30:30.559 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50592.log 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53236.log.gz 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53404.log 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50592.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.45582.log 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53362.log.gz 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53404.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50592.log.gz 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51325.log 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53404.log.gz 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48114.log 2024-03-08T08:30:30.560 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.51325.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.45582.log.gz /var/log/ceph/ceph-client.admin.45540.log 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51325.log.gz 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47382.log 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45540.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48114.log.gz 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49269.log 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45708.log 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45540.log.gz 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49269.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47382.log.gz 2024-03-08T08:30:30.561 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47214.log 2024-03-08T08:30:30.562 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49269.log.gz 2024-03-08T08:30:30.562 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49227.log 2024-03-08T08:30:30.562 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45708.log.gz 2024-03-08T08:30:30.562 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53446.log 2024-03-08T08:30:30.562 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49227.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47214.log.gz 2024-03-08T08:30:30.562 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44914.log 2024-03-08T08:30:30.562 INFO:teuthology.orchestra.run.smithi105.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49227.log.gz 2024-03-08T08:30:30.562 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53446.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44914.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53446.log.gz 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50369.log 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.1.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44914.log.gz 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53866.log 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44588.log 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50369.log.gz 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49143.log 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53866.log.gz 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44028.log 2024-03-08T08:30:30.563 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44588.log: /var/log/ceph/ceph-client.admin.49143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53740.log 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44588.log.gz 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44028.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49143.log.gz 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44028.log.gz 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53740.log: /var/log/ceph/ceph-client.admin.51445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44196.log 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53740.log.gz 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47550.log 2024-03-08T08:30:30.564 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49605.log 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50886.log 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47550.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44196.log.gz 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44238.log 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47550.log.gz 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50886.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49605.log.gz 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31037.log 2024-03-08T08:30:30.565 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50886.log.gz 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50760.log 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.tmp-client.admin.31037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50230.log 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31037.log.gz 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44238.log.gz 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44920.log 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50230.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50760.log.gz 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46044.log 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44920.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50230.log.gz 2024-03-08T08:30:30.566 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51403.log 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44920.log.gz 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52212.log 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46044.log.gz 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51403.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51403.log.gz 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50506.log 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.3.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52212.log.gz 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46002.log 2024-03-08T08:30:30.567 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51822.log 2024-03-08T08:30:30.568 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50506.log.gz 2024-03-08T08:30:30.568 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43459.log/var/log/ceph/ceph-client.admin.46002.log: 2024-03-08T08:30:30.568 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44112.log 2024-03-08T08:30:30.568 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51822.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46002.log.gz 2024-03-08T08:30:30.568 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43459.log: gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51822.log.gz 2024-03-08T08:30:30.568 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52919.log 2024-03-08T08:30:30.568 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44112.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43459.log.gz 2024-03-08T08:30:30.568 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44112.log.gz 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50548.log 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45251.log 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51780.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52919.log.gz 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50548.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49811.log 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45251.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50548.log.gz 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44364.log 2024-03-08T08:30:30.569 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45251.log.gz 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44924.log 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44364.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49811.log.gz 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44924.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44364.log.gz 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44583.log 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44924.log.gz 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47886.log 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44583.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2024-03-08T08:30:30.570 INFO:teuthology.orchestra.run.smithi105.stderr: -- /var/log/ceph/ceph-client.admin.53782.log 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47886.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44583.log.gz 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48855.log 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47886.log.gz 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49479.log 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48855.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53782.log.gz 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44922.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48855.log.gz 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46224.log 2024-03-08T08:30:30.571 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49479.log.gz 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44154.log 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46224.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44922.log.gz 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49353.log 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44154.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46224.log.gz 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45140.log 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49353.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44154.log.gz 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49353.log.gz 2024-03-08T08:30:30.572 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45498.log 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mgr.y.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45140.log.gz 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52128.log 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44406.log 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45498.log.gz 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47000.log 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50988.log 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52128.log.gz 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44406.log: /var/log/ceph/ceph-client.2.47000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49689.log 2024-03-08T08:30:30.573 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47000.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44406.log.gz 2024-03-08T08:30:30.574 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45834.log 2024-03-08T08:30:30.574 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51138.log 2024-03-08T08:30:30.574 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50988.log.gz/var/log/ceph/ceph-client.admin.49689.log: 2024-03-08T08:30:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49395.log 2024-03-08T08:30:30.574 INFO:teuthology.orchestra.run.smithi105.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49689.log.gz 2024-03-08T08:30:30.574 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48555.log 2024-03-08T08:30:30.574 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45834.log.gz 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51138.log: /var/log/ceph/ceph-client.admin.49395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51577.log 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48555.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49395.log.gz 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44923.log 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51138.log.gz 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48555.log.gz 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/ceph-client.admin.44923.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48771.log 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr: 90.9% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44923.log.gz 2024-03-08T08:30:30.575 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51577.log.gz 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2%/var/log/ceph/ceph-client.admin.43586.log: -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43818.log 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48771.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43818.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48771.log.gz 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47172.log 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43818.log.gz 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48309.log 2024-03-08T08:30:30.576 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43417.log 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47172.log.gz 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43758.log 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48309.log.gz 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53152.log 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43417.log.gz 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52690.log 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43758.log: /var/log/ceph/ceph-client.admin.53152.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43758.log.gz 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53152.log.gz 2024-03-08T08:30:30.577 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53698.log 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52044.log 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52690.log.gz 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43986.log 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53698.log.gz/var/log/ceph/ceph-client.admin.52044.log: 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52438.log 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52044.log.gz 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52480.log 2024-03-08T08:30:30.578 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52438.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43986.log.gz 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43240.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52438.log.gz 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45260.log 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43628.log 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52480.log.gz 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45792.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45260.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52877.log 2024-03-08T08:30:30.579 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43628.log.gz 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54050.log 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52877.log: 84.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.54050.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52877.log.gz 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54050.log.gz 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47718.log 2024-03-08T08:30:30.580 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52748.log 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47718.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47718.log.gz 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51948.log 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45960.log 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52748.log: /var/log/ceph/ceph-client.admin.51948.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52748.log.gz 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.51948.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45960.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45960.log.gz 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44586.log 2024-03-08T08:30:30.581 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49934.log 2024-03-08T08:30:30.582 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47760.log: /var/log/ceph/ceph-client.admin.44586.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2024-03-08T08:30:30.582 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44586.log.gz 2024-03-08T08:30:30.582 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43336.log 2024-03-08T08:30:30.582 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49934.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49934.log.gz 2024-03-08T08:30:30.582 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49101.log 2024-03-08T08:30:30.582 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48639.log 2024-03-08T08:30:30.582 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49101.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43336.log.gz 2024-03-08T08:30:30.582 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52354.log 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49101.log.gz 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53530.log 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52354.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48639.log.gz 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52354.log.gz 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48225.log 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53530.log: /var/log/ceph/ceph-client.admin.43502.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53530.log.gz 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-03-08T08:30:30.583 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47592.log 2024-03-08T08:30:30.584 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49647.log 2024-03-08T08:30:30.584 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47592.log: /var/log/ceph/ceph-client.admin.48225.log: 88.8% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48225.log.gz 2024-03-08T08:30:30.584 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47592.log.gz 2024-03-08T08:30:30.584 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48267.log 2024-03-08T08:30:30.584 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48813.log 2024-03-08T08:30:30.584 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49647.log: /var/log/ceph/ceph-client.admin.48267.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49647.log.gz 2024-03-08T08:30:30.584 INFO:teuthology.orchestra.run.smithi105.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48267.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47634.log 2024-03-08T08:30:30.584 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53656.log 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47634.log: /var/log/ceph/ceph-client.admin.48813.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48813.log.gz 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47634.log.gz 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52606.log 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50280.log 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.53656.log: /var/log/ceph/ceph-client.admin.52606.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53656.log.gz 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52606.log.gz 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49437.log 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49976.log 2024-03-08T08:30:30.585 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49437.log: /var/log/ceph/ceph-client.admin.50280.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49437.log.gz 2024-03-08T08:30:30.586 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50280.log.gz 2024-03-08T08:30:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-08T08:30:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50676.log 2024-03-08T08:30:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49976.log: /var/log/ceph/ceph-mds.a.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49976.log.gz 2024-03-08T08:30:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47298.log 2024-03-08T08:30:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50676.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50676.log.gz 2024-03-08T08:30:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52793.log 2024-03-08T08:30:30.586 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47298.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47298.log.gz 2024-03-08T08:30:30.587 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2024-03-08T08:30:30.587 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52793.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52793.log.gz 2024-03-08T08:30:30.587 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45255.log 2024-03-08T08:30:30.587 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50634.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2024-03-08T08:30:30.587 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45255.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45255.log.gz 2024-03-08T08:30:30.587 INFO:teuthology.orchestra.run.smithi105.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-08T08:30:30.588 INFO:teuthology.orchestra.run.smithi144.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-08T08:30:30.588 INFO:teuthology.orchestra.run.smithi144.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-08T08:30:30.596 INFO:teuthology.orchestra.run.smithi144.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-08T08:30:30.615 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-08T08:30:30.626 INFO:teuthology.orchestra.run.smithi105.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-08T08:30:30.663 INFO:teuthology.orchestra.run.smithi105.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-08T08:30:30.669 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-08T08:30:30.674 INFO:teuthology.orchestra.run.smithi105.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-08T08:30:30.691 INFO:teuthology.orchestra.run.smithi144.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-08T08:30:30.691 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-08T08:30:30.693 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-08T08:30:30.693 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m0.247s 2024-03-08T08:30:30.693 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m0.969s 2024-03-08T08:30:30.693 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m0.053s 2024-03-08T08:30:30.893 INFO:teuthology.orchestra.run.smithi105.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-08T08:30:31.647 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-08T08:30:31.649 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-08T08:30:31.649 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m1.233s 2024-03-08T08:30:31.649 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m2.455s 2024-03-08T08:30:31.649 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m0.198s 2024-03-08T08:30:31.650 INFO:tasks.ceph:Archiving logs... 2024-03-08T08:30:31.650 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/remote/smithi105/log 2024-03-08T08:30:31.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T08:30:32.205 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/remote/smithi144/log 2024-03-08T08:30:32.206 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T08:30:32.365 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-08T08:30:32.375 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-08T08:30:32.376 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T08:30:32.378 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T08:30:32.469 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-08T08:30:32.478 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-08T08:30:32.478 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T08:30:32.481 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T08:30:32.493 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-08T08:30:32.496 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-08T08:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T08:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-08T08:30:32.497 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 60 -900us[ -892us] +/- 139ms 2024-03-08T08:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -160us[ -151us] +/- 74ms 2024-03-08T08:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +652us[ +660us] +/- 75ms 2024-03-08T08:30:32.498 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:30:32.501 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T08:30:32.501 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-08T08:30:32.501 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 -889us[ -880us] +/- 140ms 2024-03-08T08:30:32.501 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -147us[ -138us] +/- 74ms 2024-03-08T08:30:32.501 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +649us[ +658us] +/- 75ms 2024-03-08T08:30:32.502 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T08:30:32.502 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-08T08:30:32.511 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-08T08:30:32.511 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-08T08:30:32.520 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T08:30:32.608 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T08:30:32.680 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T08:30:32.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:30:32.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-08T08:30:32.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:30:32.741 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-08T08:30:32.751 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-08T08:30:32.760 INFO:teuthology.task.internal:Duration was 643.672850 seconds 2024-03-08T08:30:32.761 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-08T08:30:32.822 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-08T08:30:32.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T08:30:32.825 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T08:30:32.865 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T08:30:32.869 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T08:30:33.267 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-08T08:30:33.267 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-03-08T08:30:33.267 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T08:30:33.326 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-03-08T08:30:33.327 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T08:30:33.355 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-08T08:30:33.355 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T08:30:33.369 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T08:30:33.424 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-08T08:30:33.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T08:30:33.438 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T08:30:33.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-08T08:30:33.571 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-08T08:30:33.571 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T08:30:33.598 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T08:30:33.628 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-08T08:30:33.638 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T08:30:33.641 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T08:30:33.664 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-03-08T08:30:33.693 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-03-08T08:30:33.709 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T08:30:33.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:30:33.735 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T08:30:33.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T08:30:33.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-08T08:30:33.773 INFO:teuthology.task.internal:Transferring archived files... 2024-03-08T08:30:33.773 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/remote/smithi105 2024-03-08T08:30:33.774 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T08:30:33.811 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/remote/smithi144 2024-03-08T08:30:33.811 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T08:30:33.846 INFO:teuthology.task.internal:Removing archive directory... 2024-03-08T08:30:33.846 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T08:30:33.849 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T08:30:33.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-08T08:30:33.907 INFO:teuthology.task.internal:Not uploading archives. 2024-03-08T08:30:33.907 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-08T08:30:33.917 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-08T08:30:33.917 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T08:30:33.919 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T08:30:33.931 INFO:teuthology.orchestra.run.smithi105.stdout: 265829 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 08:30 /home/ubuntu/cephtest 2024-03-08T08:30:33.950 INFO:teuthology.orchestra.run.smithi144.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 08:30 /home/ubuntu/cephtest 2024-03-08T08:30:33.951 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-08T08:30:33.964 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-08T08:30:33.984 INFO:teuthology.nuke:Checking targets against current locks 2024-03-08T08:30:34.010 DEBUG:teuthology.nuke:shortname: smithi105 2024-03-08T08:30:34.011 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T08:30:34.034 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 08:10:45.152895', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKTYHwn2ct8CPgIFzcYR/P//Mbb19XWmlaFVPA0yaGWofg+DIDzwNQl46giMKuzPAIQPFxspymm1LHkkDBYnbCw='} 2024-03-08T08:30:34.037 DEBUG:teuthology.nuke:shortname: smithi144 2024-03-08T08:30:34.038 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T08:30:34.055 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 08:10:45.154648', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHqi72noN/IPdTZxLSwikM+xHAuFgsGYAZR4z2w1+1k2ffghjOr7RGkIowApJ83KB6CWEXWcIZfnN+TuORTPdEY='} 2024-03-08T08:30:34.081 INFO:teuthology.orchestra.console.smithi105:Power off 2024-03-08T08:30:34.082 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T08:30:34.107 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-03-08T08:30:34.107 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T08:30:34.123 INFO:teuthology.orchestra.console.smithi144:Power off 2024-03-08T08:30:34.124 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T08:30:34.133 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-08T08:30:34.148 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-03-08T08:30:34.148 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T08:30:34.171 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-08T08:30:38.134 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T08:30:38.171 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T08:30:38.245 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-08T08:30:38.280 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-08T08:30:42.250 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T08:30:42.281 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T08:30:42.362 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-08T08:30:42.391 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-08T08:30:46.365 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T08:30:46.391 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T08:30:46.476 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-03-08T08:30:46.501 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-03-08T08:30:46.578 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-03-08T08:30:46.603 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-03-08T08:30:46.708 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin} duration: 643.6728503704071 failure_reason: 'Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters)' flavor: default owner: scheduled_jcollin@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=cd312c27103a4b5eaf38a693cff508aa status: fail success: false 2024-03-08T08:30:46.708 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T08:30:46.782 INFO:teuthology.run:FAIL